ivuorinen/f2b
1
0
Fork 0
mirror of https://github.com/ivuorinen/f2b.git synced 2026-01-26 11:24:00 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat: major infrastructure upgrades and test improvements (#62)

Browse Source 
* feat: major infrastructure upgrades and test improvements

- chore(go): upgrade Go 1.23.0 → 1.25.0 with latest dependencies
- fix(test): eliminate sudo password prompts in test environment
  * Remove F2B_TEST_SUDO usage forcing real sudo in tests
  * Refactor tests to use proper mock sudo checking
  * Remove unused setupMockRunnerForUnprivilegedTest function
- feat(docs): migrate to Serena memory system and generalize content
  * Replace TODO.md with structured .serena/memories/ system
  * Generalize documentation removing specific numerical claims
  * Add comprehensive project memories for better maintenance
- feat(build): enhance development infrastructure
  * Add Renovate integration for automated dependency updates
  * Add CodeRabbit configuration for AI code reviews
  * Update Makefile with new dependency management targets
- fix(lint): resolve all linting issues across codebase
  * Fix markdown line length violations
  * Fix YAML indentation and formatting issues
  * Ensure EditorConfig compliance (120 char limit, 2-space indent)

BREAKING CHANGE: Requires Go 1.25.0, test environment changes may affect CI

# Conflicts:
#	.go-version
#	go.sum

# Conflicts:
#	go.sum

* fix(build): move renovate comments outside shell command blocks

- Move renovate datasource comments outside of shell { } blocks
- Fixes syntax error in CI where comments inside shell blocks cause parsing issues
- All renovate functionality preserved, comments moved after command blocks
- Resolves pr-lint action failure: 'Syntax error: end of file unexpected'

* fix: address all GitHub PR review comments

- Fix critical build ldflags variable case (cmd.Version → cmd.version)
- Pin .coderabbit.yaml remote config to commit SHA for supply-chain security
- Fix Renovate JSON stabilityDays configuration (move to top-level)
- Enhance NewContextualCommand with nil-safe config and context inheritance
- Improve Makefile update-deps safety (patch-level updates, error handling)
- Generalize documentation removing hardcoded numbers for maintainability
- Replace real sudo test with proper MockRunner implementation
- Enhance path security validation with filepath.Rel and ancestor symlink resolution
- Update tool references for consistency (markdownlint-cli → markdownlint)
- Remove time-sensitive claims in documentation

* fix: correct golangci-lint installation path

Remove invalid /v2/ path from golangci-lint module reference.
The correct path is github.com/golangci/golangci-lint/cmd/golangci-lint
not github.com/golangci/golangci-lint/v2/cmd/golangci-lint

* fix: address final GitHub PR review comments

- Clarify F2B_TEST_SUDO documentation as deprecated mock-only toggle
- Remove real sudo references from testing requirements
- Fix test parallelization issue with global runner state mutation
- Add proper cleanup to restore original runner after test
- Enhance command validation with whitespace/path separator rejection
- Improve URL path handling using PathUnescape instead of QueryUnescape
- Reduce logging sensitivity by removing path details from warn messages

* fix: correct gosec installation version

Change gosec installation from @v2.24.2 to @latest to avoid
invalid version error. The v2.24.2 tag may not exist or
have version resolution issues.

* Revert "fix: correct gosec installation version"

This reverts commit cb2094aa6829ba98e1110a86e3bd48879bdb4af9.

* fix: complete version pinning and workflow cleanup

- Pin Claude Code action to v1.0.7 with commit SHA
- Remove unnecessary kics-scan ignore comment
- Add missing Renovate comments for all dev-deps
- Fix gosec version from non-existent v2.24.2 to v2.22.8
- Pin all @latest tool versions to specific releases

This completes the comprehensive version pinning strategy
for supply chain security and automated dependency management.

* chore: fix deps in Makefile

* chore(ci): commented installation of dev-deps

* chore(ci): install golangci-lint

* chore(ci): install golangci-lint

* refactor(fail2ban): harden client bootstrap and consolidate parsers

* chore(ci) reverting claude.yml to enable claude

* refactor(parser): complete ban record parser unification and TODO cleanup

✅ Unified optimized ban record parser with primary implementation
  - Consolidated ban_record_parser_optimized.go into ban_record_parser.go
  - Eliminated 497 lines of duplicate specialized code
  - Maintained all performance optimizations and backward compatibility
  - Updated all test references and method calls

✅ Validated benchmark coverage remains comprehensive
  - Line parsing, large datasets, time parsing benchmarks retained
  - Memory pooling and statistics benchmarks functional
  - Performance maintained at ~1600ns/op with 12 allocs/op

✅ Confirmed structured metrics are properly exposed
  - Cache hits/misses via ValidationCacheHits/ValidationCacheMiss
  - Parser statistics via GetStats() method (parseCount, errorCount)
  - Integration with existing metrics system complete

- Updated todo.md with completion status and technical notes
- All tests passing, 0 linting issues
- Production-ready unified parser implementation

* feat(organization): consolidate interfaces and types, fix context usage

✅ Interface Consolidation:
- Created dedicated interfaces.go for Client, Runner, SudoChecker interfaces
- Created types.go for common structs (BanRecord, LoggerInterface, etc.)
- Removed duplicate interface definitions from multiple files
- Improved code organization and maintainability

✅ Context Improvements:
- Fixed context.TODO() usage in fail2ban.go and logs.go
- Added proper context-aware functions with context.Background()
- Improved context propagation throughout the codebase

✅ Code Quality:
- All tests passing
- 0 linting issues
- No duplicate type/interface definitions
- Better separation of concerns

This establishes a cleaner foundation for further refactoring work.

* perf(config): cache regex compilation for better performance

✅ Performance Optimization:
- Moved overlongEncodingRegex compilation to package level in config_utils.go
- Eliminated repeated regex compilation in hot path of path validation
- Improves performance for Unicode encoding validation checks

✅ Code Quality:
- Better separation of concerns with module-level regex caching
- Follows Go best practices for expensive regex operations
- All tests passing, 0 linting issues

This small optimization reduces allocations and CPU usage during
path security validation operations.

* refactor(constants): consolidate format strings to constants

✅ Code Quality Improvements:
- Created PlainFormat constant to eliminate hardcoded 'plain' strings
- Updated all format string usage to use constants (PlainFormat, JSONFormat)
- Improved maintainability and reduced magic string dependencies
- Better code consistency across the cmd package

✅ Changes:
- Added PlainFormat constant in cmd/output.go
- Updated 6 files to use constants instead of hardcoded strings
- Improved documentation and comments for clarity
- All tests passing, 0 linting issues

This improves code maintainability and follows Go best practices
for string constants.

* docs(todo): update progress summary and remaining improvement opportunities

✅ Progress Summary:
- Interface consolidation and type organization completed
- Context improvements and performance optimizations implemented
- Code quality enhancements with constant consolidation
- All changes tested and validated (0 linting issues)

📋 Remaining Opportunities:
- Large file decomposition for better maintainability
- Error type improvements for better type safety
- Additional code duplication removal

The project now has a significantly cleaner and more maintainable
codebase with better separation of concerns.

* docs(packages): add comprehensive package documentation and cleanup dependencies

✅ Documentation Improvements:
- Added meaningful package documentation to 8 key files
- Enhanced cmd/ package docs for output, config, metrics, helpers, logging
- Improved fail2ban/ package docs for interfaces and types
- Better describes package purpose and functionality for developers

✅ Dependency Cleanup:
- Ran 'go mod tidy' to optimize dependencies
- Updated dependency versions where needed
- Removed unused dependencies and imports
- All dependencies verified and optimized

✅ Code Quality:
- All tests passing (100% success rate)
- 0 linting issues after improvements
- Better code maintainability and developer experience
- Improved project documentation standards

This enhances the developer experience and maintains clean,
well-documented code that follows Go best practices.

* feat(config): consolidate timeout constants and complete TODO improvements

✅ Configuration Consolidation:
- Replaced hardcoded 5*time.Second with DefaultPollingInterval constant
- Improved consistency across timeout configurations
- Better maintainability for timing-related code

✅ TODO List Progress Summary:
- Completed 9 out of 12 major improvement areas identified
- Interface consolidation, context fixes, performance optimizations ✅
- Code quality improvements, documentation enhancements ✅
- Maintenance work, dependency cleanup, configuration consolidation ✅
- All improvements tested with 100% success rate, 0 linting issues

🎯 Project Achievement:
The f2b codebase now has significantly improved maintainability,
better documentation, cleaner architecture, and follows Go best
practices throughout. Remaining work items are optional future
enhancements for a project that is already production-ready.

* feat(final): complete remaining TODO improvements - testing, deduplication, type safety

✅ Test Coverage Improvements:
- Added comprehensive tests for uncovered functions in command_test_framework.go
- Improved coverage: WithName (0% → 100%), AssertEmpty (0% → 75%), ReadStdout (0% → 25%)
- Added tests for new helper functions with full coverage
- Overall test coverage improved from 78.1% to 78.2%

✅ Code Deduplication:
- Created string processing helpers (TrimmedString, IsEmptyString, NonEmptyString)
- Added error handling helpers (WrapError, WrapErrorf) for consistent patterns
- Created command output helper (TrimmedOutput) for repeated string(bytes) operations
- Consolidated repeated validation and trimming logic

✅ Type Safety Analysis:
- Analyzed existing error handling system - already robust with ContextualError
- Confirmed structured errors with remediation hints are well-implemented
- Verified error wrapping consistency throughout codebase
- No additional improvements needed - current implementation is production-ready

🎯 Final Achievement:
- Completed 11 out of 12 TODO improvement areas (92% completion rate)
- Only optional large file decomposition remains for future consideration
- All improvements tested with 100% success rate, 0 linting issues
- Project now has exceptional code quality, maintainability, and documentation

* refactor(helpers): extract logging and environment detection module - Step 1/5

✅ Large File Decomposition - First Module Extracted:
- Created fail2ban/logging_env.go (72 lines) with focused functionality
- Extracted logging, CI detection, and test environment utilities
- Reduced fail2ban/helpers.go from 1,167 → 1,120 lines (-47 lines)

✅ Extracted Functions:
- SetLogger, getLogger, IsCI, configureCITestLogging, IsTestEnvironment
- Clean separation of concerns with dedicated logging module
- All functionality preserved with proper imports and dependencies

✅ Quality Assurance:
- All tests passing (100% success rate)
- 0 linting issues after extraction
- Zero breaking changes - backward compatibility maintained
- Proper module organization with clear package documentation

🎯 Progress: Step 1 of 5 complete for helpers.go decomposition
Next: Continue with validation, parsing, or path security modules

This demonstrates the 'one file at a time' approach working perfectly.

* docs(decomposition): document Step 2 analysis and learning from parsing extraction attempt

✅ Analysis Completed - Step 2 Learning:
- Attempted extraction of parsing utilities (ParseJailList, ParseBracketedList, etc.)
- Successfully extracted functions but discovered behavioral compatibility issues
- Test failures revealed subtle differences in output formatting and parsing logic
- Learned that exact behavioral compatibility is critical for complex function extraction

🔍 Key Insights:
- Step 1 (logging_env.go) succeeded because functions were self-contained
- Complex parsing functions have subtle interdependencies and exact behavior requirements
- Future extractions need smaller, more isolated function groups
- Behavioral compatibility testing is essential before committing extractions

📋 Refined Approach for Remaining Steps:
- Focus on smaller, self-contained function groups
- Prioritize functions with minimal behavioral complexity
- Test extensively before permanent extraction
- Consider leaving complex, interdependent functions in place

This preserves our Step 1 success while documenting valuable lessons learned.

* refactor(helpers): extract context utilities module - Step 3/5 complete

✅ Step 3 Successfully Completed:
- Created fail2ban/logging_context.go (59 lines) with focused context utilities
- Extracted WithRequestID, WithOperation, WithJail, WithIP, LoggerFromContext, GenerateRequestID
- Reduced fail2ban/helpers.go from 1,120 → 1,070 lines (-50 lines in this step)
- Total cumulative reduction: 1,167 → 1,070 lines (-97 lines extracted)

✅ Context Functions Extracted:
- All context value management functions (With* family)
- LoggerFromContext for structured logging with context fields
- GenerateRequestID for request tracing capabilities
- Small, self-contained functions with minimal dependencies

✅ Quality Results:
- 100% test success rate (all tests passing)
- 0 linting issues after extraction
- Zero breaking changes - perfect backward compatibility
- Clean separation of concerns with focused module

🎯 Progress: Step 3 of 5 complete using refined 'small extractions' strategy
Next: Continue with more small, self-contained function groups

This demonstrates the refined approach working perfectly for maintainable file decomposition.

* feat(helpers): MAJOR MILESTONE - Complete file decomposition with target exceeded!

🎯 BREAKTHROUGH ACHIEVEMENT - TARGET EXCEEDED:
- helpers.go reduced from 1,167 → 857 lines (-310 lines, 26.6% reduction)
- Target was <1,000 lines, achieved 857 lines (143 lines UNDER target!)
- Complete decomposition across 4 focused, maintainable modules

✅ Step 4 & 5 Successfully Completed:
- Step 4: security_utils.go (46 lines) - ContainsPathTraversal, GetDangerousCommandPatterns
- Step 5: validation_cache.go (180 lines) - Complete caching system with metrics

🏆 Final Module Portfolio:
- logging_env.go (73 lines) - Environment detection & logging setup
- logging_context.go (60 lines) - Context utilities & request tracing
- security_utils.go (46 lines) - Security validation & threat detection
- validation_cache.go (180 lines) - Thread-safe caching with metrics integration
- helpers.go (857 lines) - Core validation, parsing, & path utilities

✅ Perfect Quality Maintained:
- 100% test success rate across all extractions
- 0 linting issues after major decomposition
- Zero breaking changes - complete backward compatibility preserved
- Clean separation of concerns with focused, single-responsibility modules

🎊 This demonstrates successful large-scale refactoring using iterative, small-extraction approach!

* docs(todo): update with verified claims and accurate metrics

✅ Verification Completed - All Claims Validated:
- Confirmed helpers.go: 1,167 → 857 lines (26.6% reduction verified)
- Verified all 4 extracted modules exist with correct line counts:
  - logging_env.go: 73 lines ✓
  - logging_context.go: 60 lines ✓
  - security_utils.go: 46 lines ✓
  - validation_cache.go: 181 lines ✓ (corrected from 180)
- Updated current file sizes: fail2ban.go (770 lines), cmd/helpers.go (597 lines)
- Confirmed 100% test success rate and 0 linting issues
- Updated completion status: 12/12 improvement areas completed (100%)

📊 All metrics verified against actual file system and git history.
All claims in todo.md now accurately reflect the current project state.

* docs(analysis): comprehensive fresh analysis of improvement opportunities

🔍 Fresh Analysis Results - New Improvement Opportunities Identified:

✅ Code Deduplication Opportunities:
1. Command Pattern Abstraction (High Impact) - Ban/Unban 95% duplicate code
2. Test Setup Deduplication (Medium Impact) - 24+ repeated mock setup patterns
3. String Constants Consolidation - hardcoded strings across multiple files

✅ File Organization Opportunities:
4. Large Test File Decomposition - 3 files >600 lines (max 954 lines)
5. Test Coverage Improvements - target 78.2% → 85%+

✅ Code Quality Improvements:
6. Context Creation Pattern - repeated timeout context creation
7. Error Handling Consolidation - 87 error patterns analyzed

📊 Metrics Identified:
- Target: 100+ line reduction through deduplication
- Current coverage: 78.2% (cmd: 73.7%, fail2ban: 82.8%)
- 274 test functions, 171 t.Run() calls analyzed
- 7 specific improvement areas prioritized by impact

🎯 Implementation Strategy: 3-phase approach (Quick Wins → Structural → Polish)
All improvements designed to maintain 100% backward compatibility.

* refactor(cmd): implement command pattern abstraction - Phase 1 complete

✅ Phase 1 Complete: High-Impact Quick Win Achieved

🎯 Command Pattern Abstraction Successfully Implemented:
- Eliminated 95% code duplication between ban/unban commands
- Created reusable IP command pattern for consistent operations
- Established extensible architecture for future IP-based commands

📊 File Changes:
- cmd/ban.go: 76 → 19 lines (-57 lines, 75% reduction)
- cmd/unban.go: 73 → 19 lines (-54 lines, 74% reduction)
- cmd/ip_command_pattern.go: NEW (110 lines) - Reusable abstraction
- cmd/ip_processors.go: NEW (56 lines) - Processor implementations

🏆 Benefits Achieved:
✅ Zero code duplication - both commands use identical pattern
✅ Extensible architecture - new IP commands trivial to add
✅ Consistent structure - all IP operations follow same flow
✅ Maintainable codebase - pattern changes update all commands
✅ 100% backward compatibility - no breaking changes
✅ Quality maintained - 100% test pass, 0 linting issues

🎯 Next Phase: Test Setup Deduplication (24+ mock patterns to consolidate)

* docs(todo): clean progress tracker with Phase 1 completion status

* refactor(test): comprehensive test improvements and reorganization

Major test suite enhancements across multiple areas:

**Standardized Mock Setup**
- Add StandardMockSetup() helper to centralize 22 common mock patterns
- Add SetupMockEnvironmentWithStandardResponses() convenience function
- Migrate client_security_test.go to use standardized setup
- Migrate fail2ban_integration_sudo_test.go to use standardized setup
- Reduces mock configuration duplication by ~70 lines

**Test Coverage Improvements**
- Add cmd/helpers_test.go with comprehensive helper function tests
- Coverage: RequireNonEmptyArgument, FormatBannedResult, WrapError
- Coverage: NewContextualCommand, AddWatchFlags
- Improves cmd package coverage from 73.7% to 74.4%

**Test Organization**
- Extract client lifecycle tests to new client_management_test.go
- Move TestNewClient and TestSudoRequirementsChecking out of main test file
- Reduces fail2ban_fail2ban_test.go from 954 to 886 lines (-68)
- Better functional separation and maintainability

**Security Linting**
- Fix G602 gosec warning in gzip_detection.go
- Add explicit length check before slice access
- Add nosec comment with clear safety justification

**Results**
- 83.1% coverage in fail2ban package
- 74.4% coverage in cmd package
- Zero linting issues
- Significant code deduplication achieved
- All tests passing

* chore(deps): update go dependencies

* refactor: security, performance, and code quality improvements

**Security - PATH Hijacking Prevention**
- Fix TOCTOU vulnerability in client.go by capturing exec.LookPath result
- Store and use resolved absolute path instead of plain command name
- Prevents PATH manipulation between validation and execution
- Maintains MockRunner compatibility for testing

**Security - Robust Path Traversal Detection**
- Replace brittle substring checks with stdlib filepath.IsLocal validation
- Use filepath.Clean for canonicalization and additional traversal detection
- Keep minimal URL-encoded pattern checks for command validation
- Remove redundant unicode pattern checks (handled by canonicalization)
- More robust against bypasses and encoding tricks

**Security - Clean Up Dangerous Pattern Detection**
- Split GetDangerousCommandPatterns into productionPatterns and testSentinels
- Remove overly broad /etc/ pattern, replace with specific /etc/passwd and
/etc/shadow
- Eliminate duplicate entries (removed lowercase sentinel versions)
- Add comprehensive documentation explaining defensive-only purpose
- Clarify this is for log sanitization/threat detection, NOT input validation
- Add inline comments explaining each production pattern

**Memory Safety - Bounded Validation Caches**
- Add maxCacheSize limit (10000 entries) to prevent unbounded growth
- Implement automatic eviction when cache reaches 90% capacity
- Evict 25% of entries using random iteration (simple and effective)
- Protect size checks with existing mutex for thread safety
- Add debug logging for eviction events (observability)
- Update documentation explaining bounded behavior and eviction policy
- Prevents memory exhaustion in long-running processes

**Memory Safety - Remove Unsafe Shared Buffers**
- Remove unsafe shared buffers (fieldBuf, timeBuf) from BanRecordParser
- Eliminate potential race conditions on global defaultBanRecordParser
- Parser already uses goroutine-safe sync.Pool pattern for allocations
- BanRecordParser now fully goroutine-safe

**Code Quality - Concurrency Safety**
- Fix data race in ip_command_pattern.go by not mutating shared config
- Use local finalFormat variable instead of modifying config.Format in-place
- Prevents race conditions when config is shared across goroutines

**Code Quality - Logger Flexibility**
- Fix silent no-op for custom loggers in logging_env.go
- Use interface-based assertion for SetLevel instead of concrete type
- Support custom loggers that implement SetLevel(logrus.Level)
- Add debug message when log level adjustment fails (observable behavior)
- More flexible and maintainable logging configuration

**Code Quality - Error Handling Refactoring**
- Extract handleCategorizedError helper to eliminate duplication
- Consolidate pattern from HandleValidationError, HandlePermissionError, HandleSystemError
- Reduce ~90 lines to ~50 lines while preserving identical behavior
- Add errorPatternMatch type for clearer pattern-to-remediation mapping
- All handlers now use consistent lowercase pattern matching

**Code Quality - Remove Vestigial Test Instrumentation**
- Remove unused atomic counters (cacheHits, cacheMisses) from OptimizedLogProcessor
- No caching actually exists in the processor - counters were misleading
- Convert GetCacheStats and ClearCaches to no-ops for API compatibility
- Remove fail2ban_log_performance_race_test.go (136 lines testing non-existent functionality)
- Cleaner separation between production and test code

**Performance - Remove Unnecessary Allocations**
- Remove redundant slice allocation and copy in GetLogLinesOptimized
- Return collectLogLines result directly instead of making intermediate copy
- Reduces memory allocations and improves performance

**Configuration**
- Fix renovate.json regex to match version across line breaks in Makefile
- Update regex pattern to handle install line + comment line pattern
- Disable stuck linters in .mega-linter.yml (GO_GOLANGCI_LINT, JSON_V8R)

**Documentation**
- Fix nested list indentation in .serena/memories/todo.md
- Correct AGENTS.md to reference cmd/*_test.go instead of non-existent cmd.test/
- Document dangerous pattern detection purpose and usage
- Document validation cache bounds and eviction behavior

**Results**
- Zero linting issues
- All tests passing with race detector clean
- Significant code elimination (~140 lines including test cleanup)
- Improved security posture (PATH hijacking, path traversal, pattern detection)
- Improved memory safety (bounded caches, removed unsafe buffers)
- Improved performance (eliminated redundant allocations)
- Improved maintainability, consistency, and concurrency safety
- Production-ready for long-running processes

* refactor: complete deferred CodeRabbit issues and improve code quality

Implements all 6 remaining low-priority CodeRabbit review issues that were
deferred during initial development, plus additional code quality improvements.

BATCH 7 - Quick Wins (Trivial/Simple fixes):
- Fix Renovate regex pattern to match multiline comments in Makefile
* Changed from ';\\s*#' to '[\\s\\S]*?renovate:' for cross-line matching
- Add input validation to log reading functions
* Added MaxLogLinesLimit constant (100,000) for memory safety
* Validate maxLines parameter in GetLogLinesWithLimit()
* Validate maxLines parameter in GetLogLinesOptimized()
* Reject negative values and excessive limits
* Created comprehensive validation tests in logs_validation_test.go

BATCH 8 - Test Coverage Enhancement:
- Expand command_test_framework_coverage_test.go with ~225 lines of tests
* Added coverage for WithArgs, WithJSONFormat, WithSetup methods
* Added tests for Run, AssertContains, method chaining
* Added MockClientBuilder tests
* Achieved 100% coverage for key builder methods

BATCH 9 - Context Parameters (API Consistency):
- Add context.Context parameters to validation functions
* Updated ValidateLogPath(ctx, path, logDir)
* Updated ValidateClientLogPath(ctx, logDir)
* Updated ValidateClientFilterPath(ctx, filterDir)
* Updated 5 call sites across client.go and logs.go
* Enables timeout/cancellation support for file operations

BATCH 10 - Logger Interface Decoupling (Architecture):
- Decouple LoggerInterface from logrus-specific types
* Created Fields type alias to replace logrus.Fields
* Split into LoggerEntry and LoggerInterface interfaces
* Implemented adapter pattern in logrus_adapter.go (145 lines)
* Updated all code to use decoupled interfaces (7 locations)
* Removed unused logrus imports from 4 files
* Updated main.go to wrap logger with NewLogrusAdapter()
* Created comprehensive adapter tests (~280 lines)

Additional Code Quality Improvements:
- Extract duplicate error message constants (goconst compliance)
* Added ErrMaxLinesNegative constant to shared/constants.go
* Added ErrMaxLinesExceedsLimit constant to shared/constants.go
* Updated both validation sites to use constants (DRY principle)

Files Modified:
- .github/renovate.json (regex fix)
- shared/constants.go (3 new constants)
- fail2ban/types.go (decoupled interfaces)
- fail2ban/logrus_adapter.go (new adapter, 145 lines)
- fail2ban/logging_env.go (adapter initialization)
- fail2ban/logging_context.go (return type updates, removed import)
- fail2ban/logs.go (validation + constants)
- fail2ban/helpers.go (type updates, removed import)
- fail2ban/ban_record_parser.go (type updates, removed import)
- fail2ban/client.go (context parameters)
- main.go (wrap logger with adapter)
- fail2ban/logs_validation_test.go (new file, 62 lines)
- fail2ban/logrus_adapter_test.go (new file, ~280 lines)
- cmd/command_test_framework_coverage_test.go (+225 lines)
- fail2ban/fail2ban_error_handling_fix_test.go (fixed expectations)

Impact:
- Improved robustness: Input validation prevents memory exhaustion
- Better architecture: Logger interface now follows dependency inversion
- Enhanced testability: Can swap logging implementations without code changes
- API consistency: Context support enables timeout/cancellation
- Code quality: Zero duplicate constants, DRY compliance
- Tooling: Renovate can now auto-update Makefile dependencies

Verification:
✅ All tests pass: go test ./... -race -count=1
✅ Build successful: go build -o f2b .
✅ Zero linting issues
✅ goconst reports zero duplicates

* refactor: address CodeRabbit feedback on test quality and code safety

Remove redundant return statement after t.Fatal in command test framework,
preventing unreachable code warning.

Add defensive validation to NewBoundedTimeCache constructor to panic on
invalid maxSize values (≤ 0), preventing silent cache failures.

Consolidate duplicate benchmark cases in ban record parser tests from
separate original_large and optimized_large runs into single large_dataset
benchmark to reduce redundant CI time.

Refactor compatibility tests to better reflect determinism semantics by
renaming test functions (TestParserCompatibility → TestParserDeterminism),
helper functions (compareParserResults parameter names), and all
variable/parameter names from original/optimized to first/second. Updates
comments to clarify tests validate deterministic behavior across consecutive
parser runs with identical input.

Fix timestamp generation in cache eviction test to use monotonic time
increment instead of modulo arithmetic, preventing duplicate timestamps
that could mask cache bugs.

Replace hardcoded "path" log field with shared.LogFieldFile constant in
gzip detection for consistency with other logging statements in the file.

Convert unsafe type assertion to comma-ok pattern with t.Fatalf in test
helper setup to prevent panic and provide clear test failure messages.

* refactor: improve test coverage, add buffer pooling, and fix logger race condition

Add sync.Pool for duration formatting buffers in ban record parser to reduce
allocations and GC pressure during high-throughput parsing. Pooled 11-byte
buffers are reused across formatDurationOptimized calls instead of allocating
new buffers each time.

Rename TestOptimizedParserStatistics to TestParserStatistics for consistency
with determinism refactoring that removed "Optimized" naming throughout test
suite.

Strengthen cache eviction test by adding 11000 entries (CacheMaxSize + 1000)
instead of 9100 to guarantee eviction triggers during testing. Change assertion
from Less to LessOrEqual for precise boundary validation and enhance logging to
show eviction metrics (entries added, final size, max size, evicted count).

Fix race condition in logger variable access by replacing plain package-level
variable with atomic.Value for lock-free thread-safe concurrent access. Add
sync/atomic import, initialize logger via init() function using Store(), update
SetLogger to call Store() and getLogger to call Load() with type assertion.
Update ConfigureCITestLogging to use getLogger() accessor instead of direct
variable access. Eliminates data races when SetLogger is called during
concurrent logging or parallel tests while maintaining backward compatibility
and avoiding mutex overhead.

* fix: resolve CodeRabbit security issues and linting violations

Address 43 issues identified in CodeRabbit review, focusing on critical
security vulnerabilities, error handling improvements, and code quality.

Security Improvements:
- Add input validation before privilege escalation in ban/unban operations
- Re-validate paths after URL-decode and Unicode normalization to prevent
bypass attacks in path traversal protection
- Add null byte detection after path transformations
- Change test file permissions from 0644 to 0600

Error Handling:
- Convert panic-based constructors to return (value, error) tuples:
- NewBanRecordParser, NewFastTimeCache, NewBoundedTimeCache
- Add nil pointer guards in NewLogrusAdapter and SetLogger
- Improve error wrapping with proper %w format in WrapErrorf

Reliability:
- Replace time-based request IDs with UUID to prevent collisions
- Add context validation in WithRequestID and WithOperation
- Add github.com/google/uuid dependency

Testing:
- Replace os.Setenv with t.Setenv for automatic cleanup (27 instances)
- Add t.Helper() calls to test setup functions
- Rename unused function parameters to _ in test helpers
- Add comprehensive test coverage with 12 new test files

Code Quality:
- Remove TODO comments to satisfy godox linter
- Fix unused parameter warnings (revive)
- Update golangci-lint installation path in CI workflow

This resolves all 58 linting violations and fixes critical security issues
related to input validation and path traversal prevention.

* fix: resolve CodeRabbit issues and eliminate duplicate constants

Address 7 critical issues identified in CodeRabbit review and eliminate
duplicate string constants found by goconst analysis.

CodeRabbit Fixes:
- Prevent test pollution by clearing env vars before tests
(main_config_test.go)
- Fix cache eviction to check max size directly, preventing overflow under
concurrent access (fail2ban/validation_cache.go)
- Use atomic.LoadInt64 for thread-safe metric counter reads in tests
(cmd/metrics_additional_test.go)
- Close pipe writers in test goroutines to prevent ReadStdout blocking
(cmd/readstdout_additional_test.go)
- Propagate caller's context instead of using Background in command execution
(fail2ban/fail2ban.go)
- Fix BanIPWithContext assertion to accept both 0 and 1 as valid return codes
(fail2ban/helpers_validation_test.go)
- Remove unsafe test case that executed real sudo commands
(fail2ban/sudo_additional_test.go)

Code Quality:
- Replace hardcoded "all" strings with shared.AllFilter constant
- Add shared.ErrInvalidIPAddress constant for IP validation errors
- Eliminate duplicate error message strings across codebase

This resolves concurrency issues, prevents test environment pollution,
and improves code maintainability through centralized constants.

* refactor: complete context propagation and thread-safety fixes

Fix all remaining context.Background() instances where caller context was
available. This ensures timeout and cancellation signals flow through the
entire call chain from commands to client operations to validation.

Context Propagation Changes:
- fail2ban: Implement *WithContext delegation pattern for all operations
- BanIP/UnbanIP/BannedIn now delegate to *WithContext variants
- TestFilter delegates to TestFilterWithContext
- CombinedOutput/CombinedOutputWithSudo delegate to *WithContext variants
- validateFilterPath accepts context for validation chain
- All validation calls (CachedValidateIP, CachedValidateJail, etc.) use
caller ctx
- helpers: Create ValidateArgumentsWithContext and thread context through
validateSingleArgument for IP validation
- logs: streamLogFile delegates to streamLogFileWithContext
- cmd: Create ValidateIPArgumentWithContext for context-aware IP validation
- cmd: Update ip_command_pattern and testip to use *WithContext validators
- cmd: Fix banned command to pass ctx to CachedValidateJail

Thread Safety:
- metrics_additional_test: Use atomic.LoadInt64 for ValidationFailures reads
to prevent data races with atomic.AddInt64 writes

Test Framework:
- command_test_framework: Initialize Config with default timeouts to prevent
"context deadline exceeded" errors in tests that use context
This commit is contained in:
Ismo Vuorinen
2025-12-20 01:34:06 +02:00
committed by GitHub
parent 1cbb80364c
commit fa74b48038
120 changed files with 10240 additions and 4114 deletions
Download Patch File Download Diff File Expand all files Collapse all files

530
fail2ban/ban_record_parser.go
View File

@@ -2,11 +2,15 @@ package fail2ban
import (
"errors"
"fmt"
"net"
"strconv"
"strings"
"sync"
"sync/atomic"
"time"
"github.com/sirupsen/logrus"
"github.com/ivuorinen/f2b/shared"
)
// Sentinel errors for parser
@@ -16,128 +20,486 @@ var (
ErrInvalidBanTime = errors.New("invalid ban time")
)
// BanRecordParser provides optimized parsing of ban records
type BanRecordParser struct {
stringPool sync.Pool
timeCache *TimeParsingCache
// Buffer pool for duration formatting to reduce allocations
var durationBufPool = sync.Pool{
New: func() interface{} {
b := make([]byte, 0, 11)
return &b
},
}
// NewBanRecordParser creates a new optimized ban record parser
func NewBanRecordParser() *BanRecordParser {
return &BanRecordParser{
stringPool: sync.Pool{
New: func() interface{} {
s := make([]string, 0, 8) // Pre-allocate for typical field count
return &s
},
},
timeCache: defaultTimeCache,
// BoundedTimeCache provides a concurrent-safe bounded cache for parsed times
type BoundedTimeCache struct {
mu sync.RWMutex
cache map[string]time.Time
maxSize int
}
// NewBoundedTimeCache creates a new bounded time cache
func NewBoundedTimeCache(maxSize int) (*BoundedTimeCache, error) {
if maxSize <= 0 {
return nil, fmt.Errorf("BoundedTimeCache maxSize must be positive, got %d", maxSize)
}
return &BoundedTimeCache{
cache: make(map[string]time.Time),
maxSize: maxSize,
}, nil
}
// ParseBanRecordLine efficiently parses a single ban record line
// Load retrieves a cached time value
func (btc *BoundedTimeCache) Load(key string) (time.Time, bool) {
btc.mu.RLock()
t, ok := btc.cache[key]
btc.mu.RUnlock()
return t, ok
}
// Store caches a time value with automatic eviction when threshold is reached
func (btc *BoundedTimeCache) Store(key string, value time.Time) {
btc.mu.Lock()
defer btc.mu.Unlock()
// Check if we need to evict before adding
if len(btc.cache) >= int(float64(btc.maxSize)*shared.CacheEvictionThreshold) {
btc.evictEntries()
}
btc.cache[key] = value
}
// evictEntries removes entries to bring cache back to target size
// Caller must hold btc.mu lock
func (btc *BoundedTimeCache) evictEntries() {
targetSize := int(float64(len(btc.cache)) * (1.0 - shared.CacheEvictionRate))
count := 0
for key := range btc.cache {
if len(btc.cache) <= targetSize {
break
}
delete(btc.cache, key)
count++
}
getLogger().WithFields(Fields{
"evicted": count,
"remaining": len(btc.cache),
"max_size": btc.maxSize,
}).Debug("Evicted time cache entries")
}
// Size returns the current number of entries in the cache
func (btc *BoundedTimeCache) Size() int {
btc.mu.RLock()
defer btc.mu.RUnlock()
return len(btc.cache)
}
// BanRecordParser provides high-performance parsing of ban records
type BanRecordParser struct {
// Pools for zero-allocation parsing (goroutine-safe)
stringPool sync.Pool
recordPool sync.Pool
timeCache *FastTimeCache
// Statistics for monitoring
parseCount int64
errorCount int64
}
// FastTimeCache provides ultra-fast time parsing with minimal allocations
type FastTimeCache struct {
layout string
parseCache *BoundedTimeCache // Bounded cache with max 10k entries
stringPool sync.Pool
}
// NewBanRecordParser creates a new high-performance ban record parser
func NewBanRecordParser() (*BanRecordParser, error) {
timeCache, err := NewFastTimeCache(shared.TimeFormat)
if err != nil {
return nil, fmt.Errorf("failed to create parser: %w", err)
}
parser := &BanRecordParser{
timeCache: timeCache,
}
// String pool for reusing field slices
parser.stringPool = sync.Pool{
New: func() interface{} {
s := make([]string, 0, 16)
return &s
},
}
// Record pool for reusing BanRecord objects
parser.recordPool = sync.Pool{
New: func() interface{} {
return &BanRecord{}
},
}
return parser, nil
}
// NewFastTimeCache creates an optimized time cache
func NewFastTimeCache(layout string) (*FastTimeCache, error) {
parseCache, err := NewBoundedTimeCache(shared.CacheMaxSize)
if err != nil {
return nil, fmt.Errorf("failed to create time cache: %w", err)
}
cache := &FastTimeCache{
layout: layout,
parseCache: parseCache,
}
cache.stringPool = sync.Pool{
New: func() interface{} {
b := make([]byte, 0, 32)
return &b
},
}
return cache, nil
}
// ParseTimeOptimized parses time with minimal allocations
func (ftc *FastTimeCache) ParseTimeOptimized(timeStr string) (time.Time, error) {
// Fast path: check cache
if cached, ok := ftc.parseCache.Load(timeStr); ok {
return cached, nil
}
// Parse and cache - only cache successful parses
t, err := time.Parse(ftc.layout, timeStr)
if err == nil {
ftc.parseCache.Store(timeStr, t)
}
return t, err
}
// BuildTimeStringOptimized builds time string with zero allocations using byte buffer
func (ftc *FastTimeCache) BuildTimeStringOptimized(dateStr, timeStr string) string {
bufPtr := ftc.stringPool.Get().(*[]byte)
buf := *bufPtr
defer func() {
buf = buf[:0] // Reset buffer
*bufPtr = buf
ftc.stringPool.Put(bufPtr)
}()
// Calculate required capacity
totalLen := len(dateStr) + 1 + len(timeStr)
if cap(buf) < totalLen {
buf = make([]byte, 0, totalLen)
*bufPtr = buf
}
// Build string using byte operations
buf = append(buf, dateStr...)
buf = append(buf, ' ')
buf = append(buf, timeStr...)
// Convert to string - Go compiler will optimize this
return string(buf)
}
// ParseBanRecordLine parses a single line with maximum performance
func (brp *BanRecordParser) ParseBanRecordLine(line, jail string) (*BanRecord, error) {
line = strings.TrimSpace(line)
if line == "" {
// Fast path: check for empty line
if len(line) == 0 {
return nil, ErrEmptyLine
}
// Get pooled slice for fields
// Trim whitespace in-place if needed
line = fastTrimSpace(line)
if len(line) == 0 {
return nil, ErrEmptyLine
}
// Get pooled field slice
fieldsPtr := brp.stringPool.Get().(*[]string)
fields := *fieldsPtr
fields := (*fieldsPtr)[:0] // Reset slice but keep capacity
defer func() {
if len(fields) > 0 {
resetFields := fields[:0]
*fieldsPtr = resetFields
brp.stringPool.Put(fieldsPtr) // Reset slice and return to pool
}
*fieldsPtr = fields[:0]
brp.stringPool.Put(fieldsPtr)
}()
// Parse fields more efficiently
fields = strings.Fields(line)
// Fast field parsing - avoid strings.Fields allocation
fields = fastSplitFields(line, fields)
if len(fields) < 1 {
return nil, ErrInsufficientFields
}
ip := fields[0]
if len(fields) >= 8 {
// Format: IP BANNED_DATE BANNED_TIME + UNBAN_DATE UNBAN_TIME
bannedStr := brp.timeCache.BuildTimeString(fields[1], fields[2])
unbanStr := brp.timeCache.BuildTimeString(fields[4], fields[5])
tBan, err := brp.timeCache.ParseTime(bannedStr)
if err != nil {
getLogger().WithFields(logrus.Fields{
"jail": jail,
"ip": ip,
"bannedStr": bannedStr,
}).Warnf("Failed to parse ban time: %v", err)
// Skip this entry if we can't parse the ban time (original behavior)
return nil, ErrInvalidBanTime
}
tUnban, err := brp.timeCache.ParseTime(unbanStr)
if err != nil {
getLogger().WithFields(logrus.Fields{
"jail": jail,
"ip": ip,
"unbanStr": unbanStr,
}).Warnf("Failed to parse unban time: %v", err)
// Use current time as fallback for unban time calculation
tUnban = time.Now().Add(DefaultBanDuration) // Assume 24h remaining
}
rem := tUnban.Unix() - time.Now().Unix()
if rem < 0 {
rem = 0
}
return &BanRecord{
Jail: jail,
IP: ip,
BannedAt: tBan,
Remaining: FormatDuration(rem),
}, nil
// Validate jail name for path traversal
if jail == "" || strings.ContainsAny(jail, "/\\") || strings.Contains(jail, "..") {
return nil, fmt.Errorf("invalid jail name: contains unsafe characters")
}
// Fallback for simpler format
return &BanRecord{
Jail: jail,
IP: ip,
BannedAt: time.Now(),
Remaining: "unknown",
}, nil
// Validate IP address format
if fields[0] != "" && net.ParseIP(fields[0]) == nil {
return nil, fmt.Errorf(shared.ErrInvalidIPAddress, fields[0])
}
// Get pooled record
record := brp.recordPool.Get().(*BanRecord)
defer brp.recordPool.Put(record)
// Reset record fields
*record = BanRecord{
Jail: jail,
IP: fields[0],
}
// Fast path for full format (8+ fields)
if len(fields) >= 8 {
return brp.parseFullFormat(fields, record)
}
// Fallback for simple format
record.BannedAt = time.Now()
record.Remaining = shared.UnknownValue
// Return a copy since we're pooling the original
result := &BanRecord{
Jail: record.Jail,
IP: record.IP,
BannedAt: record.BannedAt,
Remaining: record.Remaining,
}
return result, nil
}
// ParseBanRecords parses multiple ban record lines efficiently
// parseFullFormat handles the full 8-field format efficiently
func (brp *BanRecordParser) parseFullFormat(fields []string, record *BanRecord) (*BanRecord, error) {
// Build time strings efficiently
bannedStr := brp.timeCache.BuildTimeStringOptimized(fields[1], fields[2])
unbanStr := brp.timeCache.BuildTimeStringOptimized(fields[4], fields[5])
// Parse ban time
tBan, err := brp.timeCache.ParseTimeOptimized(bannedStr)
if err != nil {
getLogger().WithFields(Fields{
"jail": record.Jail,
"ip": record.IP,
"bannedStr": bannedStr,
}).Warnf("Failed to parse ban time: %v", err)
return nil, ErrInvalidBanTime
}
// Parse unban time with fallback
tUnban, err := brp.timeCache.ParseTimeOptimized(unbanStr)
if err != nil {
getLogger().WithFields(Fields{
"jail": record.Jail,
"ip": record.IP,
"unbanStr": unbanStr,
}).Warnf("Failed to parse unban time: %v", err)
tUnban = time.Now().Add(shared.DefaultBanDuration) // 24h fallback
}
// Calculate remaining time efficiently
now := time.Now()
rem := tUnban.Unix() - now.Unix()
if rem < 0 {
rem = 0
}
// Set parsed values
record.BannedAt = tBan
record.Remaining = formatDurationOptimized(rem)
// Return a copy since we're pooling the original
result := &BanRecord{
Jail: record.Jail,
IP: record.IP,
BannedAt: record.BannedAt,
Remaining: record.Remaining,
}
return result, nil
}
// ParseBanRecords parses multiple records with maximum efficiency
func (brp *BanRecordParser) ParseBanRecords(output string, jail string) ([]BanRecord, error) {
lines := strings.Split(strings.TrimSpace(output), "\n")
records := make([]BanRecord, 0, len(lines)) // Pre-allocate based on line count
if len(output) == 0 {
return []BanRecord{}, nil
}
// Fast line splitting without allocation where possible
lines := fastSplitLines(strings.TrimSpace(output))
records := make([]BanRecord, 0, len(lines))
for _, line := range lines {
record, err := brp.ParseBanRecordLine(line, jail)
if err != nil {
// Skip lines with parsing errors (empty lines, insufficient fields, invalid times)
if len(line) == 0 {
continue
}
record, err := brp.ParseBanRecordLine(line, jail)
if err != nil {
atomic.AddInt64(&brp.errorCount, 1)
continue // Skip invalid lines
}
if record != nil {
records = append(records, *record)
atomic.AddInt64(&brp.parseCount, 1)
}
}
return records, nil
}
// Global parser instance for reuse
var defaultBanRecordParser = NewBanRecordParser()
// GetStats returns parsing statistics
func (brp *BanRecordParser) GetStats() (parseCount, errorCount int64) {
return atomic.LoadInt64(&brp.parseCount), atomic.LoadInt64(&brp.errorCount)
}
// ParseBanRecordLineOptimized parses a ban record line using the default parser
// fastTrimSpace trims whitespace efficiently
func fastTrimSpace(s string) string {
start := 0
end := len(s)
// Trim leading whitespace
for start < end && (s[start] == ' ' || s[start] == '\t' || s[start] == '\n' || s[start] == '\r') {
start++
}
// Trim trailing whitespace
for end > start && (s[end-1] == ' ' || s[end-1] == '\t' || s[end-1] == '\n' || s[end-1] == '\r') {
end--
}
return s[start:end]
}
// fastSplitFields splits on whitespace efficiently, reusing provided slice
func fastSplitFields(s string, fields []string) []string {
fields = fields[:0] // Reset but keep capacity
start := 0
for i := 0; i < len(s); i++ {
if s[i] == ' ' || s[i] == '\t' {
if i > start {
fields = append(fields, s[start:i])
}
// Skip consecutive whitespace
for i < len(s) && (s[i] == ' ' || s[i] == '\t') {
i++
}
start = i
i-- // Compensate for loop increment
}
}
// Add final field if any
if start < len(s) {
fields = append(fields, s[start:])
}
return fields
}
// fastSplitLines splits on newlines efficiently
func fastSplitLines(s string) []string {
if len(s) == 0 {
return nil
}
lines := make([]string, 0, strings.Count(s, "\n")+1)
start := 0
for i := 0; i < len(s); i++ {
if s[i] == '\n' {
lines = append(lines, s[start:i])
start = i + 1
}
}
// Add final line if any
if start < len(s) {
lines = append(lines, s[start:])
}
return lines
}
// formatDurationOptimized formats duration efficiently in DD:HH:MM:SS format to match original
func formatDurationOptimized(sec int64) string {
days := sec / shared.SecondsPerDay
h := (sec % shared.SecondsPerDay) / shared.SecondsPerHour
m := (sec % shared.SecondsPerHour) / shared.SecondsPerMinute
s := sec % shared.SecondsPerMinute
// Get buffer from pool to reduce allocations
bufPtr := durationBufPool.Get().(*[]byte)
buf := (*bufPtr)[:0]
defer func() {
*bufPtr = buf[:0]
durationBufPool.Put(bufPtr)
}()
// Format days (2 digits)
if days < 10 {
buf = append(buf, '0')
}
buf = strconv.AppendInt(buf, days, 10)
buf = append(buf, ':')
// Format hours (2 digits)
if h < 10 {
buf = append(buf, '0')
}
buf = strconv.AppendInt(buf, h, 10)
buf = append(buf, ':')
// Format minutes (2 digits)
if m < 10 {
buf = append(buf, '0')
}
buf = strconv.AppendInt(buf, m, 10)
buf = append(buf, ':')
// Format seconds (2 digits)
if s < 10 {
buf = append(buf, '0')
}
buf = strconv.AppendInt(buf, s, 10)
return string(buf)
}
// Global parser instance for reuse
var defaultBanRecordParser = mustCreateParser()
// mustCreateParser creates a parser or panics (used for global init only)
func mustCreateParser() *BanRecordParser {
parser, err := NewBanRecordParser()
if err != nil {
panic(fmt.Sprintf("failed to create default ban record parser: %v", err))
}
return parser
}
// ParseBanRecordLineOptimized parses a ban record line using the default parser.
func ParseBanRecordLineOptimized(line, jail string) (*BanRecord, error) {
return defaultBanRecordParser.ParseBanRecordLine(line, jail)
}
// ParseBanRecordsOptimized parses multiple ban records using the default parser
// ParseBanRecordsOptimized parses multiple ban records using the default parser.
func ParseBanRecordsOptimized(output, jail string) ([]BanRecord, error) {
return defaultBanRecordParser.ParseBanRecords(output, jail)
}
// ParseBanRecordsUltraOptimized is an alias for backward compatibility
func ParseBanRecordsUltraOptimized(output, jail string) ([]BanRecord, error) {
return ParseBanRecordsOptimized(output, jail)
}
// ParseBanRecordLineUltraOptimized is an alias for backward compatibility
func ParseBanRecordLineUltraOptimized(line, jail string) (*BanRecord, error) {
return ParseBanRecordLineOptimized(line, jail)
}

381
fail2ban/ban_record_parser_optimized.go
View File

@@ -1,381 +0,0 @@
package fail2ban
import (
"strconv"
"strings"
"sync"
"sync/atomic"
"time"
"github.com/sirupsen/logrus"
)
// OptimizedBanRecordParser provides high-performance parsing of ban records
type OptimizedBanRecordParser struct {
// Pre-allocated buffers for zero-allocation parsing
fieldBuf []string
timeBuf []byte
stringPool sync.Pool
recordPool sync.Pool
timeCache *FastTimeCache
// Statistics for monitoring
parseCount int64
errorCount int64
}
// FastTimeCache provides ultra-fast time parsing with minimal allocations
type FastTimeCache struct {
layout string
layoutBytes []byte
parseCache sync.Map
stringPool sync.Pool
}
// NewOptimizedBanRecordParser creates a new high-performance ban record parser
func NewOptimizedBanRecordParser() *OptimizedBanRecordParser {
parser := &OptimizedBanRecordParser{
fieldBuf: make([]string, 0, 16), // Pre-allocate for max expected fields
timeBuf: make([]byte, 0, 32), // Pre-allocate for time string building
timeCache: NewFastTimeCache("2006-01-02 15:04:05"),
}
// String pool for reusing field slices
parser.stringPool = sync.Pool{
New: func() interface{} {
s := make([]string, 0, 16)
return &s
},
}
// Record pool for reusing BanRecord objects
parser.recordPool = sync.Pool{
New: func() interface{} {
return &BanRecord{}
},
}
return parser
}
// NewFastTimeCache creates an optimized time cache
func NewFastTimeCache(layout string) *FastTimeCache {
cache := &FastTimeCache{
layout: layout,
layoutBytes: []byte(layout),
}
cache.stringPool = sync.Pool{
New: func() interface{} {
b := make([]byte, 0, 32)
return &b
},
}
return cache
}
// ParseTimeOptimized parses time with minimal allocations
func (ftc *FastTimeCache) ParseTimeOptimized(timeStr string) (time.Time, error) {
// Fast path: check cache
if cached, ok := ftc.parseCache.Load(timeStr); ok {
return cached.(time.Time), nil
}
// Parse and cache - only cache successful parses
t, err := time.Parse(ftc.layout, timeStr)
if err == nil {
ftc.parseCache.Store(timeStr, t)
}
return t, err
}
// BuildTimeStringOptimized builds time string with zero allocations using byte buffer
func (ftc *FastTimeCache) BuildTimeStringOptimized(dateStr, timeStr string) string {
bufPtr := ftc.stringPool.Get().(*[]byte)
buf := *bufPtr
defer func() {
buf = buf[:0] // Reset buffer
*bufPtr = buf
ftc.stringPool.Put(bufPtr)
}()
// Calculate required capacity
totalLen := len(dateStr) + 1 + len(timeStr)
if cap(buf) < totalLen {
buf = make([]byte, 0, totalLen)
*bufPtr = buf
}
// Build string using byte operations
buf = append(buf, dateStr...)
buf = append(buf, ' ')
buf = append(buf, timeStr...)
// Convert to string - Go compiler will optimize this
return string(buf)
}
// ParseBanRecordLineOptimized parses a single line with maximum performance
func (obp *OptimizedBanRecordParser) ParseBanRecordLineOptimized(line, jail string) (*BanRecord, error) {
// Fast path: check for empty line
if len(line) == 0 {
return nil, ErrEmptyLine
}
// Trim whitespace in-place if needed
line = fastTrimSpace(line)
if len(line) == 0 {
return nil, ErrEmptyLine
}
// Get pooled field slice
fieldsPtr := obp.stringPool.Get().(*[]string)
fields := (*fieldsPtr)[:0] // Reset slice but keep capacity
defer func() {
*fieldsPtr = fields[:0]
obp.stringPool.Put(fieldsPtr)
}()
// Fast field parsing - avoid strings.Fields allocation
fields = fastSplitFields(line, fields)
if len(fields) < 1 {
return nil, ErrInsufficientFields
}
// Get pooled record
record := obp.recordPool.Get().(*BanRecord)
defer obp.recordPool.Put(record)
// Reset record fields
*record = BanRecord{
Jail: jail,
IP: fields[0],
}
// Fast path for full format (8+ fields)
if len(fields) >= 8 {
return obp.parseFullFormat(fields, record)
}
// Fallback for simple format
record.BannedAt = time.Now()
record.Remaining = "unknown"
// Return a copy since we're pooling the original
result := &BanRecord{
Jail: record.Jail,
IP: record.IP,
BannedAt: record.BannedAt,
Remaining: record.Remaining,
}
return result, nil
}
// parseFullFormat handles the full 8-field format efficiently
func (obp *OptimizedBanRecordParser) parseFullFormat(fields []string, record *BanRecord) (*BanRecord, error) {
// Build time strings efficiently
bannedStr := obp.timeCache.BuildTimeStringOptimized(fields[1], fields[2])
unbanStr := obp.timeCache.BuildTimeStringOptimized(fields[4], fields[5])
// Parse ban time
tBan, err := obp.timeCache.ParseTimeOptimized(bannedStr)
if err != nil {
getLogger().WithFields(logrus.Fields{
"jail": record.Jail,
"ip": record.IP,
"bannedStr": bannedStr,
}).Warnf("Failed to parse ban time: %v", err)
return nil, ErrInvalidBanTime
}
// Parse unban time with fallback
tUnban, err := obp.timeCache.ParseTimeOptimized(unbanStr)
if err != nil {
getLogger().WithFields(logrus.Fields{
"jail": record.Jail,
"ip": record.IP,
"unbanStr": unbanStr,
}).Warnf("Failed to parse unban time: %v", err)
tUnban = time.Now().Add(DefaultBanDuration) // 24h fallback
}
// Calculate remaining time efficiently
now := time.Now()
rem := tUnban.Unix() - now.Unix()
if rem < 0 {
rem = 0
}
// Set parsed values
record.BannedAt = tBan
record.Remaining = formatDurationOptimized(rem)
// Return a copy since we're pooling the original
result := &BanRecord{
Jail: record.Jail,
IP: record.IP,
BannedAt: record.BannedAt,
Remaining: record.Remaining,
}
return result, nil
}
// ParseBanRecordsOptimized parses multiple records with maximum efficiency
func (obp *OptimizedBanRecordParser) ParseBanRecordsOptimized(output string, jail string) ([]BanRecord, error) {
if len(output) == 0 {
return []BanRecord{}, nil
}
// Fast line splitting without allocation where possible
lines := fastSplitLines(strings.TrimSpace(output))
records := make([]BanRecord, 0, len(lines))
for _, line := range lines {
if len(line) == 0 {
continue
}
record, err := obp.ParseBanRecordLineOptimized(line, jail)
if err != nil {
atomic.AddInt64(&obp.errorCount, 1)
continue // Skip invalid lines
}
if record != nil {
records = append(records, *record)
atomic.AddInt64(&obp.parseCount, 1)
}
}
return records, nil
}
// fastTrimSpace trims whitespace efficiently
func fastTrimSpace(s string) string {
start := 0
end := len(s)
// Trim leading whitespace
for start < end && (s[start] == ' ' || s[start] == '\t' || s[start] == '\n' || s[start] == '\r') {
start++
}
// Trim trailing whitespace
for end > start && (s[end-1] == ' ' || s[end-1] == '\t' || s[end-1] == '\n' || s[end-1] == '\r') {
end--
}
return s[start:end]
}
// fastSplitFields splits on whitespace efficiently, reusing provided slice
func fastSplitFields(s string, fields []string) []string {
fields = fields[:0] // Reset but keep capacity
start := 0
for i := 0; i < len(s); i++ {
if s[i] == ' ' || s[i] == '\t' {
if i > start {
fields = append(fields, s[start:i])
}
// Skip consecutive whitespace
for i < len(s) && (s[i] == ' ' || s[i] == '\t') {
i++
}
start = i
i-- // Compensate for loop increment
}
}
// Add final field if any
if start < len(s) {
fields = append(fields, s[start:])
}
return fields
}
// fastSplitLines splits on newlines efficiently
func fastSplitLines(s string) []string {
if len(s) == 0 {
return nil
}
lines := make([]string, 0, strings.Count(s, "\n")+1)
start := 0
for i := 0; i < len(s); i++ {
if s[i] == '\n' {
lines = append(lines, s[start:i])
start = i + 1
}
}
// Add final line if any
if start < len(s) {
lines = append(lines, s[start:])
}
return lines
}
// formatDurationOptimized formats duration efficiently in DD:HH:MM:SS format to match original
func formatDurationOptimized(sec int64) string {
days := sec / SecondsPerDay
h := (sec % SecondsPerDay) / SecondsPerHour
m := (sec % SecondsPerHour) / SecondsPerMinute
s := sec % SecondsPerMinute
// Pre-allocate buffer for DD:HH:MM:SS format (11 chars)
buf := make([]byte, 0, 11)
// Format days (2 digits)
if days < 10 {
buf = append(buf, '0')
}
buf = strconv.AppendInt(buf, days, 10)
buf = append(buf, ':')
// Format hours (2 digits)
if h < 10 {
buf = append(buf, '0')
}
buf = strconv.AppendInt(buf, h, 10)
buf = append(buf, ':')
// Format minutes (2 digits)
if m < 10 {
buf = append(buf, '0')
}
buf = strconv.AppendInt(buf, m, 10)
buf = append(buf, ':')
// Format seconds (2 digits)
if s < 10 {
buf = append(buf, '0')
}
buf = strconv.AppendInt(buf, s, 10)
return string(buf)
}
// GetStats returns parsing statistics
func (obp *OptimizedBanRecordParser) GetStats() (parseCount, errorCount int64) {
return atomic.LoadInt64(&obp.parseCount), atomic.LoadInt64(&obp.errorCount)
}
// Global optimized parser instance
var optimizedBanRecordParser = NewOptimizedBanRecordParser()
// ParseBanRecordLineUltraOptimized parses a ban record line using the optimized parser
func ParseBanRecordLineUltraOptimized(line, jail string) (*BanRecord, error) {
return optimizedBanRecordParser.ParseBanRecordLineOptimized(line, jail)
}
// ParseBanRecordsUltraOptimized parses multiple ban records using the optimized parser
func ParseBanRecordsUltraOptimized(output, jail string) ([]BanRecord, error) {
return optimizedBanRecordParser.ParseBanRecordsOptimized(output, jail)
}

118
fail2ban/client.go
View File

@@ -4,65 +4,20 @@ import (
"context"
"errors"
"fmt"
"os"
"os/exec"
"strings"
"time"
"github.com/ivuorinen/f2b/shared"
)
// Client defines the interface for interacting with Fail2Ban.
// Implementations must provide all core operations for jail and ban management.
type Client interface {
// ListJails returns all available Fail2Ban jails.
ListJails() ([]string, error)
// StatusAll returns the status output for all jails.
StatusAll() (string, error)
// StatusJail returns the status output for a specific jail.
StatusJail(string) (string, error)
// BanIP bans the given IP in the specified jail. Returns 0 if banned, 1 if already banned.
BanIP(ip, jail string) (int, error)
// UnbanIP unbans the given IP in the specified jail. Returns 0 if unbanned, 1 if already unbanned.
UnbanIP(ip, jail string) (int, error)
// BannedIn returns the list of jails in which the IP is currently banned.
BannedIn(ip string) ([]string, error)
// GetBanRecords returns ban records for the specified jails.
GetBanRecords(jails []string) ([]BanRecord, error)
// GetLogLines returns log lines filtered by jail and/or IP.
GetLogLines(jail, ip string) ([]string, error)
// ListFilters returns the available Fail2Ban filters.
ListFilters() ([]string, error)
// TestFilter runs fail2ban-regex for the given filter.
TestFilter(filter string) (string, error)
// Context-aware versions for timeout and cancellation support
ListJailsWithContext(ctx context.Context) ([]string, error)
StatusAllWithContext(ctx context.Context) (string, error)
StatusJailWithContext(ctx context.Context, jail string) (string, error)
BanIPWithContext(ctx context.Context, ip, jail string) (int, error)
UnbanIPWithContext(ctx context.Context, ip, jail string) (int, error)
BannedInWithContext(ctx context.Context, ip string) ([]string, error)
GetBanRecordsWithContext(ctx context.Context, jails []string) ([]BanRecord, error)
GetLogLinesWithContext(ctx context.Context, jail, ip string) ([]string, error)
ListFiltersWithContext(ctx context.Context) ([]string, error)
TestFilterWithContext(ctx context.Context, filter string) (string, error)
}
// RealClient is the default implementation of Client, using the local fail2ban-client binary.
type RealClient struct {
Path string // Path to fail2ban-client
Path string // Command used to invoke fail2ban-client
Jails []string
LogDir string
FilterDir string
}
// BanRecord represents a single ban entry with jail, IP, ban time, and remaining duration.
type BanRecord struct {
Jail string
IP string
BannedAt time.Time
Remaining string
}
// NewClient initializes a RealClient, verifying the environment and fail2ban-client availability.
// It checks for fail2ban-client in PATH, ensures the service is running, checks sudo privileges,
// and loads available jails. Returns an error if fail2ban is not available, not running, or
@@ -76,66 +31,63 @@ func NewClient(logDir, filterDir string) (*RealClient, error) {
// and loads available jails. Returns an error if fail2ban is not available, not running, or
// user lacks sudo privileges.
func NewClientWithContext(ctx context.Context, logDir, filterDir string) (*RealClient, error) {
// Check sudo privileges first (skip in test environment unless forced)
if !IsTestEnvironment() || os.Getenv("F2B_TEST_SUDO") == "true" {
// Check sudo privileges first (skip in test environment)
if !IsTestEnvironment() {
if err := CheckSudoRequirements(); err != nil {
return nil, err
}
}
path, err := exec.LookPath(Fail2BanClientCommand)
// Resolve the absolute path to prevent PATH hijacking
resolvedPath, err := exec.LookPath(shared.Fail2BanClientCommand)
if err != nil {
// Check if we have a mock runner set up
if _, ok := GetRunner().(*MockRunner); !ok {
return nil, fmt.Errorf("%s not found in PATH", Fail2BanClientCommand)
return nil, fmt.Errorf("%s not found in PATH", shared.Fail2BanClientCommand)
}
path = Fail2BanClientCommand // Use mock path
}
if logDir == "" {
logDir = DefaultLogDir
}
if filterDir == "" {
filterDir = DefaultFilterDir
// For mock runner, use the plain command name
resolvedPath = shared.Fail2BanClientCommand
}
// Validate log directory
logAllowedPaths := GetLogAllowedPaths()
logConfig := PathSecurityConfig{
AllowedBasePaths: logAllowedPaths,
MaxPathLength: 4096,
AllowSymlinks: false,
ResolveSymlinks: true,
if logDir == "" {
logDir = shared.DefaultLogDir
}
validatedLogDir, err := validatePathWithSecurity(logDir, logConfig)
if filterDir == "" {
filterDir = shared.DefaultFilterDir
}
// Validate log directory using centralized helper with context
validatedLogDir, err := ValidateClientLogPath(ctx, logDir)
if err != nil {
return nil, fmt.Errorf("invalid log directory: %w", err)
}
// Validate filter directory
filterAllowedPaths := GetFilterAllowedPaths()
filterConfig := PathSecurityConfig{
AllowedBasePaths: filterAllowedPaths,
MaxPathLength: 4096,
AllowSymlinks: false,
ResolveSymlinks: true,
}
validatedFilterDir, err := validatePathWithSecurity(filterDir, filterConfig)
// Validate filter directory using centralized helper with context
validatedFilterDir, err := ValidateClientFilterPath(ctx, filterDir)
if err != nil {
return nil, fmt.Errorf("invalid filter directory: %w", err)
return nil, fmt.Errorf("%s: %w", shared.ErrInvalidFilterDirectory, err)
}
rc := &RealClient{Path: path, LogDir: validatedLogDir, FilterDir: validatedFilterDir}
rc := &RealClient{
Path: resolvedPath, // Use resolved absolute path
LogDir: validatedLogDir,
FilterDir: validatedFilterDir,
}
// Version check - use sudo if needed with context
out, err := RunnerCombinedOutputWithSudoContext(ctx, path, "-V")
out, err := RunnerCombinedOutputWithSudoContext(ctx, rc.Path, "-V")
if err != nil {
return nil, fmt.Errorf("version check failed: %w", err)
}
if CompareVersions(strings.TrimSpace(string(out)), "0.11.0") < 0 {
return nil, fmt.Errorf("fail2ban >=0.11.0 required, got %s", out)
rawVersion := strings.TrimSpace(string(out))
parsedVersion, err := ExtractFail2BanVersion(rawVersion)
if err != nil {
return nil, fmt.Errorf("failed to parse fail2ban version: %w", err)
}
if CompareVersions(parsedVersion, "0.11.0") < 0 {
return nil, fmt.Errorf("fail2ban >=0.11.0 required, got %s", rawVersion)
}
// Ping - use sudo if needed with context
if _, err := RunnerCombinedOutputWithSudoContext(ctx, path, "ping"); err != nil {
if _, err := RunnerCombinedOutputWithSudoContext(ctx, rc.Path, "ping"); err != nil {
return nil, errors.New("fail2ban service not running")
}
jails, err := rc.fetchJailsWithContext(ctx)

65
fail2ban/client_management_test.go Normal file
View File

@@ -0,0 +1,65 @@
package fail2ban
import (
"strings"
"testing"
"github.com/ivuorinen/f2b/shared"
)
func TestNewClient(t *testing.T) {
// Test normal client creation (in test environment, sudo checking is skipped)
t.Run("normal client creation", func(t *testing.T) {
// Set up mock environment with standard responses
_, cleanup := SetupMockEnvironmentWithStandardResponses(t)
defer cleanup()
client, err := NewClient(shared.DefaultLogDir, shared.DefaultFilterDir)
if err != nil {
t.Fatalf("expected no error, got %v", err)
}
if client == nil {
t.Fatal("expected client to be non-nil")
}
})
}
func TestSudoRequirementsChecking(t *testing.T) {
tests := []struct {
name string
hasPrivileges bool
expectError bool
errorContains string
}{
{
name: "with sudo privileges",
hasPrivileges: true,
expectError: false,
},
{
name: "without sudo privileges",
hasPrivileges: false,
expectError: true,
errorContains: "fail2ban operations require sudo privileges",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Set up mock environment
_, cleanup := SetupMockEnvironmentWithSudo(t, tt.hasPrivileges)
defer cleanup()
// Test the sudo checking function directly
err := CheckSudoRequirements()
AssertError(t, err, tt.expectError, tt.name)
if tt.expectError {
if tt.errorContains != "" && err != nil && !strings.Contains(err.Error(), tt.errorContains) {
t.Errorf("expected error to contain %q, got %q", tt.errorContains, err.Error())
}
return
}
})
}
}

53
fail2ban/client_security_test.go
View File

@@ -3,25 +3,15 @@ package fail2ban
import (
"strings"
"testing"
"github.com/ivuorinen/f2b/shared"
)
func TestNewClientPathTraversalProtection(t *testing.T) {
// Enable test mode
t.Setenv("F2B_TEST_SUDO", "true")
// Set up mock environment
_, cleanup := SetupMockEnvironment(t)
// Set up mock environment with standard responses
_, cleanup := SetupMockEnvironmentWithStandardResponses(t)
defer cleanup()
// Get the mock runner and configure additional responses
mock := GetRunner().(*MockRunner)
mock.SetResponse("fail2ban-client -V", []byte("Fail2Ban v0.11.2"))
mock.SetResponse("sudo fail2ban-client -V", []byte("Fail2Ban v0.11.2"))
mock.SetResponse("fail2ban-client ping", []byte("pong"))
mock.SetResponse("sudo fail2ban-client ping", []byte("pong"))
mock.SetResponse("fail2ban-client status", []byte("Status\n|- Number of jail: 1\n`- Jail list: sshd"))
mock.SetResponse("sudo fail2ban-client status", []byte("Status\n|- Number of jail: 1\n`- Jail list: sshd"))
tests := []struct {
name string
logDir string
@@ -168,22 +158,10 @@ func TestNewClientPathTraversalProtection(t *testing.T) {
}
func TestNewClientDefaultPathValidation(t *testing.T) {
// Enable test mode
t.Setenv("F2B_TEST_SUDO", "true")
// Set up mock environment
_, cleanup := SetupMockEnvironment(t)
// Set up mock environment with standard responses
_, cleanup := SetupMockEnvironmentWithStandardResponses(t)
defer cleanup()
// Get the mock runner and configure additional responses
mock := GetRunner().(*MockRunner)
mock.SetResponse("fail2ban-client -V", []byte("Fail2Ban v0.11.2"))
mock.SetResponse("sudo fail2ban-client -V", []byte("Fail2Ban v0.11.2"))
mock.SetResponse("fail2ban-client ping", []byte("pong"))
mock.SetResponse("sudo fail2ban-client ping", []byte("pong"))
mock.SetResponse("fail2ban-client status", []byte("Status\n|- Number of jail: 1\n`- Jail list: sshd"))
mock.SetResponse("sudo fail2ban-client status", []byte("Status\n|- Number of jail: 1\n`- Jail list: sshd"))
// Test with empty paths (should use defaults and validate them)
client, err := NewClient("", "")
if err != nil {
@@ -191,12 +169,23 @@ func TestNewClientDefaultPathValidation(t *testing.T) {
}
// Verify defaults were applied
if client.LogDir != DefaultLogDir {
t.Errorf("expected LogDir to be %s, got %s", DefaultLogDir, client.LogDir)
if client.LogDir != shared.DefaultLogDir {
t.Errorf("expected LogDir to be %s, got %s", shared.DefaultLogDir, client.LogDir)
}
if client.FilterDir != DefaultFilterDir {
t.Errorf("expected FilterDir to be %s, got %s", DefaultFilterDir, client.FilterDir)
if client.FilterDir != shared.DefaultFilterDir {
if resolved, err := resolveAncestorSymlinks(shared.DefaultFilterDir, true); err == nil {
if client.FilterDir != resolved {
t.Errorf(
"expected FilterDir to be %s or %s, got %s",
shared.DefaultFilterDir,
resolved,
client.FilterDir,
)
}
} else {
t.Errorf("expected FilterDir to be %s, got %s", shared.DefaultFilterDir, client.FilterDir)
}
}
}

608
fail2ban/client_withcontext_test.go Normal file
View File

@@ -0,0 +1,608 @@
package fail2ban
import (
"context"
"errors"
"os"
"path/filepath"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// setupBasicMockResponses sets up the basic responses needed for client initialization
func setupBasicMockResponses(m *MockRunner) {
m.SetResponse("fail2ban-client -V", []byte("Fail2Ban v0.11.0"))
m.SetResponse("sudo fail2ban-client -V", []byte("Fail2Ban v0.11.0"))
m.SetResponse("fail2ban-client ping", []byte("Server replied: pong"))
m.SetResponse("sudo fail2ban-client ping", []byte("Server replied: pong"))
// NewClient calls fetchJailsWithContext which runs status
m.SetResponse("fail2ban-client status", []byte("Status\n|- Number of jail: 2\n`- Jail list: sshd, apache"))
m.SetResponse("sudo fail2ban-client status", []byte("Status\n|- Number of jail: 2\n`- Jail list: sshd, apache"))
}
// TestListJailsWithContext tests jail listing with context
func TestListJailsWithContext(t *testing.T) {
tests := []struct {
name string
setupMock func(*MockRunner)
timeout time.Duration
expectError bool
expectJails []string
}{
{
name: "successful jail listing",
setupMock: func(m *MockRunner) {
setupBasicMockResponses(m)
},
timeout: 5 * time.Second,
expectError: false,
expectJails: []string{"sshd", "apache"}, // From setupBasicMockResponses
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
mock := NewMockRunner()
tt.setupMock(mock)
SetRunner(mock)
client, err := NewClient("/var/log", "/etc/fail2ban/filter.d")
require.NoError(t, err)
ctx, cancel := context.WithTimeout(context.Background(), tt.timeout)
defer cancel()
if tt.timeout == 1*time.Nanosecond {
time.Sleep(2 * time.Millisecond) // Ensure timeout
}
jails, err := client.ListJailsWithContext(ctx)
if tt.expectError {
assert.Error(t, err)
} else {
assert.NoError(t, err)
assert.Equal(t, tt.expectJails, jails)
}
})
}
}
// TestStatusAllWithContext tests status all with context
func TestStatusAllWithContext(t *testing.T) {
tests := []struct {
name string
setupMock func(*MockRunner)
timeout time.Duration
expectError bool
expectContains string
}{
{
name: "successful status all",
setupMock: func(m *MockRunner) {
setupBasicMockResponses(m)
m.SetResponse("fail2ban-client status", []byte("Status\n|- Number of jail: 1\n`- Jail list: sshd"))
m.SetResponse("sudo fail2ban-client status", []byte("Status\n|- Number of jail: 1\n`- Jail list: sshd"))
},
timeout: 5 * time.Second,
expectError: false,
expectContains: "Status",
},
{
name: "context timeout",
setupMock: func(m *MockRunner) {
setupBasicMockResponses(m)
m.SetResponse("fail2ban-client status", []byte("Status\n|- Number of jail: 1\n`- Jail list: sshd"))
m.SetResponse("sudo fail2ban-client status", []byte("Status\n|- Number of jail: 1\n`- Jail list: sshd"))
},
timeout: 1 * time.Nanosecond,
expectError: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
mock := NewMockRunner()
tt.setupMock(mock)
SetRunner(mock)
client, err := NewClient("/var/log", "/etc/fail2ban/filter.d")
require.NoError(t, err)
ctx, cancel := context.WithTimeout(context.Background(), tt.timeout)
defer cancel()
if tt.timeout == 1*time.Nanosecond {
time.Sleep(2 * time.Millisecond)
}
status, err := client.StatusAllWithContext(ctx)
if tt.expectError {
assert.Error(t, err)
} else {
assert.NoError(t, err)
assert.Contains(t, status, tt.expectContains)
}
})
}
}
// TestStatusJailWithContext tests status jail with context
func TestStatusJailWithContext(t *testing.T) {
tests := []struct {
name string
jail string
setupMock func(*MockRunner)
timeout time.Duration
expectError bool
expectContains string
}{
{
name: "successful status jail",
jail: "sshd",
setupMock: func(m *MockRunner) {
setupBasicMockResponses(m)
m.SetResponse(
"fail2ban-client status sshd",
[]byte("Status for the jail: sshd\n|- Filter\n`- Currently banned: 0"),
)
m.SetResponse(
"sudo fail2ban-client status sshd",
[]byte("Status for the jail: sshd\n|- Filter\n`- Currently banned: 0"),
)
},
timeout: 5 * time.Second,
expectError: false,
expectContains: "sshd",
},
{
name: "invalid jail name",
jail: "invalid@jail",
setupMock: func(m *MockRunner) {
setupBasicMockResponses(m)
// Validation will fail before command execution
},
timeout: 5 * time.Second,
expectError: true,
},
{
name: "context timeout",
jail: "sshd",
setupMock: func(m *MockRunner) {
setupBasicMockResponses(m)
m.SetResponse(
"fail2ban-client status sshd",
[]byte("Status for the jail: sshd\n|- Filter\n`- Currently banned: 0"),
)
m.SetResponse(
"sudo fail2ban-client status sshd",
[]byte("Status for the jail: sshd\n|- Filter\n`- Currently banned: 0"),
)
},
timeout: 1 * time.Nanosecond,
expectError: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
mock := NewMockRunner()
tt.setupMock(mock)
SetRunner(mock)
client, err := NewClient("/var/log", "/etc/fail2ban/filter.d")
require.NoError(t, err)
ctx, cancel := context.WithTimeout(context.Background(), tt.timeout)
defer cancel()
if tt.timeout == 1*time.Nanosecond {
time.Sleep(2 * time.Millisecond)
}
status, err := client.StatusJailWithContext(ctx, tt.jail)
if tt.expectError {
assert.Error(t, err)
} else {
assert.NoError(t, err)
if tt.expectContains != "" {
assert.Contains(t, status, tt.expectContains)
}
}
})
}
}
// TestUnbanIPWithContext tests unban IP with context
func TestUnbanIPWithContext(t *testing.T) {
tests := []struct {
name string
ip string
jail string
setupMock func(*MockRunner)
timeout time.Duration
expectError bool
expectCode int
}{
{
name: "successful unban",
ip: "192.168.1.100",
jail: "sshd",
setupMock: func(m *MockRunner) {
setupBasicMockResponses(m)
m.SetResponse("fail2ban-client set sshd unbanip 192.168.1.100", []byte("0"))
m.SetResponse("sudo fail2ban-client set sshd unbanip 192.168.1.100", []byte("0"))
},
timeout: 5 * time.Second,
expectError: false,
expectCode: 0,
},
{
name: "already unbanned",
ip: "192.168.1.100",
jail: "sshd",
setupMock: func(m *MockRunner) {
setupBasicMockResponses(m)
m.SetResponse("fail2ban-client set sshd unbanip 192.168.1.100", []byte("1"))
m.SetResponse("sudo fail2ban-client set sshd unbanip 192.168.1.100", []byte("1"))
},
timeout: 5 * time.Second,
expectError: false,
expectCode: 1,
},
{
name: "invalid IP address",
ip: "invalid-ip",
jail: "sshd",
setupMock: func(m *MockRunner) {
setupBasicMockResponses(m)
// Validation will fail before command execution
},
timeout: 5 * time.Second,
expectError: true,
},
{
name: "invalid jail name",
ip: "192.168.1.100",
jail: "invalid@jail",
setupMock: func(m *MockRunner) {
setupBasicMockResponses(m)
// Validation will fail before command execution
},
timeout: 5 * time.Second,
expectError: true,
},
{
name: "context timeout",
ip: "192.168.1.100",
jail: "sshd",
setupMock: func(m *MockRunner) {
setupBasicMockResponses(m)
m.SetResponse("fail2ban-client set sshd unbanip 192.168.1.100", []byte("0"))
m.SetResponse("sudo fail2ban-client set sshd unbanip 192.168.1.100", []byte("0"))
},
timeout: 1 * time.Nanosecond,
expectError: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
mock := NewMockRunner()
tt.setupMock(mock)
SetRunner(mock)
client, err := NewClient("/var/log", "/etc/fail2ban/filter.d")
require.NoError(t, err)
ctx, cancel := context.WithTimeout(context.Background(), tt.timeout)
defer cancel()
if tt.timeout == 1*time.Nanosecond {
time.Sleep(2 * time.Millisecond)
}
code, err := client.UnbanIPWithContext(ctx, tt.ip, tt.jail)
if tt.expectError {
assert.Error(t, err)
} else {
assert.NoError(t, err)
assert.Equal(t, tt.expectCode, code)
}
})
}
}
// TestListFiltersWithContext tests filter listing with context
func TestListFiltersWithContext(t *testing.T) {
tests := []struct {
name string
setupMock func(*MockRunner)
setupEnv func()
timeout time.Duration
expectError bool
expectFilters []string
}{
{
name: "successful filter listing",
setupMock: func(m *MockRunner) {
setupBasicMockResponses(m)
// Mock responses not needed - uses file system
},
setupEnv: func() {
// Client will use default filter directory
},
timeout: 5 * time.Second,
expectError: false,
expectFilters: nil, // Will depend on actual filter directory
},
{
name: "context timeout",
setupMock: func(m *MockRunner) {
setupBasicMockResponses(m)
// Not applicable for file system operation
},
setupEnv: func() {
// No setup needed
},
timeout: 1 * time.Nanosecond,
expectError: true, // Context check happens first
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
mock := NewMockRunner()
tt.setupMock(mock)
SetRunner(mock)
tt.setupEnv()
client, err := NewClient("/var/log", "/etc/fail2ban/filter.d")
require.NoError(t, err)
ctx, cancel := context.WithTimeout(context.Background(), tt.timeout)
defer cancel()
if tt.timeout == 1*time.Nanosecond {
time.Sleep(2 * time.Millisecond)
}
filters, err := client.ListFiltersWithContext(ctx)
if tt.expectError {
assert.Error(t, err)
} else {
// May error if directory doesn't exist, which is fine in tests
if err == nil {
assert.NotNil(t, filters)
}
}
})
}
}
// TestTestFilterWithContext tests filter testing with context
func TestTestFilterWithContext(t *testing.T) {
// Enable dev paths to allow temporary directory
t.Setenv("ALLOW_DEV_PATHS", "1")
// Create temporary filter directory
tmpDir := t.TempDir()
filterContent := `[Definition]
failregex = ^.* Failed .* for .* from <HOST>
logpath = /var/log/auth.log
`
err := os.WriteFile(filepath.Join(tmpDir, "sshd.conf"), []byte(filterContent), 0600)
require.NoError(t, err)
tests := []struct {
name string
filter string
setupMock func(*MockRunner)
timeout time.Duration
expectError bool
}{
{
name: "successful filter test",
filter: "sshd",
setupMock: func(m *MockRunner) {
setupBasicMockResponses(m)
m.SetResponse(
"fail2ban-regex /var/log/auth.log "+filepath.Join(tmpDir, "sshd.conf"),
[]byte("Success: 0 matches"),
)
m.SetResponse(
"sudo fail2ban-regex /var/log/auth.log "+filepath.Join(tmpDir, "sshd.conf"),
[]byte("Success: 0 matches"),
)
},
timeout: 5 * time.Second,
expectError: false,
},
{
name: "invalid filter name",
filter: "invalid@filter",
setupMock: func(m *MockRunner) {
setupBasicMockResponses(m)
// Validation will fail before command execution
},
timeout: 5 * time.Second,
expectError: true,
},
{
name: "context timeout",
filter: "sshd",
setupMock: func(m *MockRunner) {
setupBasicMockResponses(m)
m.SetResponse(
"fail2ban-regex /var/log/auth.log "+filepath.Join(tmpDir, "sshd.conf"),
[]byte("Success: 0 matches"),
)
m.SetResponse(
"sudo fail2ban-regex /var/log/auth.log "+filepath.Join(tmpDir, "sshd.conf"),
[]byte("Success: 0 matches"),
)
},
timeout: 1 * time.Nanosecond,
expectError: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
mock := NewMockRunner()
tt.setupMock(mock)
SetRunner(mock)
client, err := NewClient("/var/log", tmpDir)
require.NoError(t, err)
ctx, cancel := context.WithTimeout(context.Background(), tt.timeout)
defer cancel()
if tt.timeout == 1*time.Nanosecond {
time.Sleep(2 * time.Millisecond)
}
result, err := client.TestFilterWithContext(ctx, tt.filter)
if tt.expectError {
assert.Error(t, err)
} else {
assert.NoError(t, err)
assert.NotEmpty(t, result)
}
})
}
}
// TestWithContextCancellation tests that all WithContext functions respect cancellation
func TestWithContextCancellation(t *testing.T) {
mock := NewMockRunner()
setupBasicMockResponses(mock)
SetRunner(mock)
client, err := NewClient("/var/log", "/etc/fail2ban/filter.d")
require.NoError(t, err)
// Create canceled context
ctx, cancel := context.WithCancel(context.Background())
cancel() // Cancel immediately
// Note: ListJailsWithContext and ListFiltersWithContext are too fast to be canceled
// as they return cached data or read from filesystem. Only testing I/O operations.
t.Run("StatusAllWithContext respects cancellation", func(t *testing.T) {
_, err := client.StatusAllWithContext(ctx)
assert.Error(t, err)
assert.True(t, errors.Is(err, context.Canceled) || isContextError(err))
})
t.Run("StatusJailWithContext respects cancellation", func(t *testing.T) {
_, err := client.StatusJailWithContext(ctx, "sshd")
assert.Error(t, err)
assert.True(t, errors.Is(err, context.Canceled) || isContextError(err))
})
t.Run("UnbanIPWithContext respects cancellation", func(t *testing.T) {
_, err := client.UnbanIPWithContext(ctx, "192.168.1.100", "sshd")
assert.Error(t, err)
assert.True(t, errors.Is(err, context.Canceled) || isContextError(err))
})
}
// TestWithContextDeadline tests that all WithContext functions respect deadlines
func TestWithContextDeadline(t *testing.T) {
mock := NewMockRunner()
setupBasicMockResponses(mock)
SetRunner(mock)
client, err := NewClient("/var/log", "/etc/fail2ban/filter.d")
require.NoError(t, err)
// Create context with very short deadline
ctx, cancel := context.WithTimeout(context.Background(), 1*time.Nanosecond)
defer cancel()
// Ensure timeout
time.Sleep(2 * time.Millisecond)
// Note: ListJailsWithContext, ListFiltersWithContext, and TestFilterWithContext
// are too fast to timeout as they return cached data or read from filesystem.
// Only testing I/O operations that make network/command calls.
tests := []struct {
name string
fn func() error
}{
{
name: "StatusAllWithContext",
fn: func() error {
_, err := client.StatusAllWithContext(ctx)
return err
},
},
{
name: "StatusJailWithContext",
fn: func() error {
_, err := client.StatusJailWithContext(ctx, "sshd")
return err
},
},
{
name: "UnbanIPWithContext",
fn: func() error {
_, err := client.UnbanIPWithContext(ctx, "192.168.1.100", "sshd")
return err
},
},
}
for _, tt := range tests {
t.Run(tt.name+" respects deadline", func(t *testing.T) {
err := tt.fn()
assert.Error(t, err)
assert.True(t, errors.Is(err, context.DeadlineExceeded) || isContextError(err))
})
}
}
// TestWithContextValidation tests that validation happens before context usage
func TestWithContextValidation(t *testing.T) {
mock := NewMockRunner()
setupBasicMockResponses(mock)
SetRunner(mock)
client, err := NewClient("/var/log", "/etc/fail2ban/filter.d")
require.NoError(t, err)
ctx := context.Background()
t.Run("StatusJailWithContext validates jail name", func(t *testing.T) {
_, err := client.StatusJailWithContext(ctx, "invalid@jail")
assert.Error(t, err)
assert.Contains(t, err.Error(), "invalid")
})
t.Run("UnbanIPWithContext validates IP", func(t *testing.T) {
_, err := client.UnbanIPWithContext(ctx, "invalid-ip", "sshd")
assert.Error(t, err)
})
t.Run("UnbanIPWithContext validates jail", func(t *testing.T) {
_, err := client.UnbanIPWithContext(ctx, "192.168.1.100", "invalid@jail")
assert.Error(t, err)
assert.Contains(t, err.Error(), "invalid")
})
t.Run("TestFilterWithContext validates filter", func(t *testing.T) {
_, err := client.TestFilterWithContext(ctx, "invalid@filter")
assert.Error(t, err)
})
}

404
fail2ban/fail2ban.go
View File

@@ -12,24 +12,13 @@ import (
"sort"
"strings"
"sync"
"github.com/ivuorinen/f2b/shared"
)
const (
// DefaultLogDir is the default directory for fail2ban logs
DefaultLogDir = "/var/log"
// DefaultFilterDir is the default directory for fail2ban filters
DefaultFilterDir = "/etc/fail2ban/filter.d"
// AllFilter represents all jails/IPs filter
AllFilter = "all"
// DefaultMaxFileSize is the default maximum file size for log reading (100MB)
DefaultMaxFileSize = 100 * 1024 * 1024
// DefaultLogLinesLimit is the default limit for log lines returned
DefaultLogLinesLimit = 1000
)
var logDir = DefaultLogDir // base directory for fail2ban logs
var logDirMu sync.RWMutex // protects logDir from concurrent access
var filterDir = DefaultFilterDir
var logDir = shared.DefaultLogDir // base directory for fail2ban logs
var logDirMu sync.RWMutex // protects logDir from concurrent access
var filterDir = shared.DefaultFilterDir
var filterDirMu sync.RWMutex // protects filterDir from concurrent access
// GetFilterDir returns the current filter directory path.
@@ -60,84 +49,41 @@ func SetFilterDir(dir string) {
filterDir = dir
}
// Runner executes system commands.
// Implementations may use sudo or other mechanisms as needed.
type Runner interface {
CombinedOutput(name string, args ...string) ([]byte, error)
CombinedOutputWithSudo(name string, args ...string) ([]byte, error)
// Context-aware versions for timeout and cancellation support
CombinedOutputWithContext(ctx context.Context, name string, args ...string) ([]byte, error)
CombinedOutputWithSudoContext(ctx context.Context, name string, args ...string) ([]byte, error)
}
// OSRunner runs commands locally.
type OSRunner struct{}
// CombinedOutput executes a command without sudo.
func (r *OSRunner) CombinedOutput(name string, args ...string) ([]byte, error) {
// Validate command for security
if err := CachedValidateCommand(name); err != nil {
return nil, fmt.Errorf("command validation failed: %w", err)
}
// Validate arguments for security
if err := ValidateArguments(args); err != nil {
return nil, fmt.Errorf("argument validation failed: %w", err)
}
return exec.Command(name, args...).CombinedOutput()
return r.CombinedOutputWithContext(context.Background(), name, args...)
}
// CombinedOutputWithContext executes a command without sudo with context support.
func (r *OSRunner) CombinedOutputWithContext(ctx context.Context, name string, args ...string) ([]byte, error) {
// Validate command for security
if err := CachedValidateCommand(name); err != nil {
return nil, fmt.Errorf("command validation failed: %w", err)
if err := CachedValidateCommand(ctx, name); err != nil {
return nil, fmt.Errorf(shared.ErrCommandValidationFailed, err)
}
// Validate arguments for security
if err := ValidateArguments(args); err != nil {
return nil, fmt.Errorf("argument validation failed: %w", err)
if err := ValidateArgumentsWithContext(ctx, args); err != nil {
return nil, fmt.Errorf(shared.ErrArgumentValidationFailed, err)
}
return exec.CommandContext(ctx, name, args...).CombinedOutput()
}
// CombinedOutputWithSudo executes a command with sudo if needed.
func (r *OSRunner) CombinedOutputWithSudo(name string, args ...string) ([]byte, error) {
// Validate command for security
if err := CachedValidateCommand(name); err != nil {
return nil, fmt.Errorf("command validation failed: %w", err)
}
// Validate arguments for security
if err := ValidateArguments(args); err != nil {
return nil, fmt.Errorf("argument validation failed: %w", err)
}
checker := GetSudoChecker()
// If already root, no need for sudo
if checker.IsRoot() {
return exec.Command(name, args...).CombinedOutput()
}
// If command requires sudo and user has privileges, use sudo
if RequiresSudo(name, args...) && checker.HasSudoPrivileges() {
sudoArgs := append([]string{name}, args...)
// #nosec G204 - This is a legitimate use case for executing fail2ban-client with sudo
// The command name and arguments are validated by ValidateCommand() and RequiresSudo()
return exec.Command("sudo", sudoArgs...).CombinedOutput()
}
// Otherwise run without sudo
return exec.Command(name, args...).CombinedOutput()
return r.CombinedOutputWithSudoContext(context.Background(), name, args...)
}
// CombinedOutputWithSudoContext executes a command with sudo if needed, with context support.
func (r *OSRunner) CombinedOutputWithSudoContext(ctx context.Context, name string, args ...string) ([]byte, error) {
// Validate command for security
if err := CachedValidateCommand(name); err != nil {
return nil, fmt.Errorf("command validation failed: %w", err)
if err := CachedValidateCommand(ctx, name); err != nil {
return nil, fmt.Errorf(shared.ErrCommandValidationFailed, err)
}
// Validate arguments for security
if err := ValidateArguments(args); err != nil {
return nil, fmt.Errorf("argument validation failed: %w", err)
if err := ValidateArgumentsWithContext(ctx, args); err != nil {
return nil, fmt.Errorf(shared.ErrArgumentValidationFailed, err)
}
checker := GetSudoChecker()
@@ -152,7 +98,7 @@ func (r *OSRunner) CombinedOutputWithSudoContext(ctx context.Context, name strin
sudoArgs := append([]string{name}, args...)
// #nosec G204 - This is a legitimate use case for executing fail2ban-client with sudo
// The command name and arguments are validated by ValidateCommand() and RequiresSudo()
return exec.CommandContext(ctx, "sudo", sudoArgs...).CombinedOutput()
return exec.CommandContext(ctx, shared.SudoCommand, sudoArgs...).CombinedOutput()
}
// Otherwise run without sudo
@@ -191,9 +137,7 @@ func GetRunner() Runner {
func RunnerCombinedOutput(name string, args ...string) ([]byte, error) {
timer := NewTimedOperation("RunnerCombinedOutput", name, args...)
globalRunnerManager.mu.RLock()
runner := globalRunnerManager.runner
globalRunnerManager.mu.RUnlock()
runner := GetRunner()
output, err := runner.CombinedOutput(name, args...)
timer.Finish(err)
@@ -206,9 +150,7 @@ func RunnerCombinedOutput(name string, args ...string) ([]byte, error) {
func RunnerCombinedOutputWithSudo(name string, args ...string) ([]byte, error) {
timer := NewTimedOperation("RunnerCombinedOutputWithSudo", name, args...)
globalRunnerManager.mu.RLock()
runner := globalRunnerManager.runner
globalRunnerManager.mu.RUnlock()
runner := GetRunner()
output, err := runner.CombinedOutputWithSudo(name, args...)
timer.Finish(err)
@@ -221,9 +163,7 @@ func RunnerCombinedOutputWithSudo(name string, args ...string) ([]byte, error) {
func RunnerCombinedOutputWithContext(ctx context.Context, name string, args ...string) ([]byte, error) {
timer := NewTimedOperation("RunnerCombinedOutputWithContext", name, args...)
globalRunnerManager.mu.RLock()
runner := globalRunnerManager.runner
globalRunnerManager.mu.RUnlock()
runner := GetRunner()
output, err := runner.CombinedOutputWithContext(ctx, name, args...)
timer.FinishWithContext(ctx, err)
@@ -236,9 +176,7 @@ func RunnerCombinedOutputWithContext(ctx context.Context, name string, args ...s
func RunnerCombinedOutputWithSudoContext(ctx context.Context, name string, args ...string) ([]byte, error) {
timer := NewTimedOperation("RunnerCombinedOutputWithSudoContext", name, args...)
globalRunnerManager.mu.RLock()
runner := globalRunnerManager.runner
globalRunnerManager.mu.RUnlock()
runner := GetRunner()
output, err := runner.CombinedOutputWithSudoContext(ctx, name, args...)
timer.FinishWithContext(ctx, err)
@@ -266,15 +204,27 @@ func NewMockRunner() *MockRunner {
// CombinedOutput returns a mocked response or error for a command.
func (m *MockRunner) CombinedOutput(name string, args ...string) ([]byte, error) {
// Prevent actual sudo execution in tests
if name == "sudo" {
key := name + " " + strings.Join(args, " ")
if name == shared.SudoCommand {
m.mu.Lock()
defer m.mu.Unlock()
m.CallLog = append(m.CallLog, key)
if err, exists := m.Errors[key]; exists {
return nil, err
}
if response, exists := m.Responses[key]; exists {
return response, nil
}
return nil, fmt.Errorf("sudo should not be called directly in tests")
}
m.mu.Lock()
defer m.mu.Unlock()
key := name + " " + strings.Join(args, " ")
m.CallLog = append(m.CallLog, key)
if err, exists := m.Errors[key]; exists {
@@ -376,7 +326,7 @@ func (m *MockRunner) CombinedOutputWithSudoContext(ctx context.Context, name str
func (c *RealClient) fetchJailsWithContext(ctx context.Context) ([]string, error) {
currentRunner := GetRunner()
out, err := currentRunner.CombinedOutputWithSudoContext(ctx, c.Path, "status")
out, err := currentRunner.CombinedOutputWithSudoContext(ctx, c.Path, shared.CommandArgStatus)
if err != nil {
return nil, err
}
@@ -386,87 +336,30 @@ func (c *RealClient) fetchJailsWithContext(ctx context.Context) ([]string, error
// StatusAll returns the status of all fail2ban jails.
func (c *RealClient) StatusAll() (string, error) {
currentRunner := GetRunner()
out, err := currentRunner.CombinedOutputWithSudo(c.Path, "status")
out, err := currentRunner.CombinedOutputWithSudo(c.Path, shared.CommandArgStatus)
return string(out), err
}
// StatusJail returns the status of a specific fail2ban jail.
func (c *RealClient) StatusJail(j string) (string, error) {
currentRunner := GetRunner()
out, err := currentRunner.CombinedOutputWithSudo(c.Path, "status", j)
out, err := currentRunner.CombinedOutputWithSudo(c.Path, shared.CommandArgStatus, j)
return string(out), err
}
// BanIP bans an IP address in the specified jail and returns the ban status code.
func (c *RealClient) BanIP(ip, jail string) (int, error) {
if err := CachedValidateIP(ip); err != nil {
return 0, err
}
if err := CachedValidateJail(jail); err != nil {
return 0, err
}
// Check if jail exists
if err := ValidateJailExists(jail, c.Jails); err != nil {
return 0, err
}
currentRunner := GetRunner()
out, err := currentRunner.CombinedOutputWithSudo(c.Path, "set", jail, "banip", ip)
if err != nil {
return 0, fmt.Errorf("failed to ban IP %s in jail %s: %w", ip, jail, err)
}
code := strings.TrimSpace(string(out))
if code == Fail2BanStatusSuccess {
return 0, nil
}
if code == Fail2BanStatusAlreadyProcessed {
return 1, nil
}
return 0, fmt.Errorf("unexpected output from fail2ban-client: %s", code)
return c.BanIPWithContext(context.Background(), ip, jail)
}
// UnbanIP unbans an IP address from the specified jail and returns the unban status code.
func (c *RealClient) UnbanIP(ip, jail string) (int, error) {
if err := CachedValidateIP(ip); err != nil {
return 0, err
}
if err := CachedValidateJail(jail); err != nil {
return 0, err
}
// Check if jail exists
if err := ValidateJailExists(jail, c.Jails); err != nil {
return 0, err
}
currentRunner := GetRunner()
out, err := currentRunner.CombinedOutputWithSudo(c.Path, "set", jail, "unbanip", ip)
if err != nil {
return 0, fmt.Errorf("failed to unban IP %s in jail %s: %w", ip, jail, err)
}
code := strings.TrimSpace(string(out))
if code == Fail2BanStatusSuccess {
return 0, nil
}
if code == Fail2BanStatusAlreadyProcessed {
return 1, nil
}
return 0, fmt.Errorf("unexpected output from fail2ban-client: %s", code)
return c.UnbanIPWithContext(context.Background(), ip, jail)
}
// BannedIn returns a list of jails where the specified IP address is currently banned.
func (c *RealClient) BannedIn(ip string) ([]string, error) {
if err := CachedValidateIP(ip); err != nil {
return nil, err
}
currentRunner := GetRunner()
out, err := currentRunner.CombinedOutputWithSudo(c.Path, "banned", ip)
if err != nil {
return nil, fmt.Errorf("failed to check if IP %s is banned: %w", ip, err)
}
return ParseBracketedList(string(out)), nil
return c.BannedInWithContext(context.Background(), ip)
}
// GetBanRecords retrieves ban records for the specified jails.
@@ -477,15 +370,13 @@ func (c *RealClient) GetBanRecords(jails []string) ([]BanRecord, error) {
// getBanRecordsInternal is the internal implementation with context support
func (c *RealClient) getBanRecordsInternal(ctx context.Context, jails []string) ([]BanRecord, error) {
var toQuery []string
if len(jails) == 1 && (jails[0] == AllFilter || jails[0] == "") {
if len(jails) == 1 && (jails[0] == shared.AllFilter || jails[0] == "") {
toQuery = c.Jails
} else {
toQuery = jails
}
globalRunnerManager.mu.RLock()
currentRunner := globalRunnerManager.runner
globalRunnerManager.mu.RUnlock()
currentRunner := GetRunner()
// Use parallel processing for multiple jails
allRecords, err := ProcessJailsParallel(
@@ -495,14 +386,14 @@ func (c *RealClient) getBanRecordsInternal(ctx context.Context, jails []string)
out, err := currentRunner.CombinedOutputWithSudoContext(
operationCtx,
c.Path,
"get",
shared.ActionGet,
jail,
"banip",
shared.ActionBanIP,
"--with-time",
)
if err != nil {
// Log error but continue processing (backward compatibility)
getLogger().WithError(err).WithField("jail", jail).
getLogger().WithError(err).WithField(string(shared.ContextKeyJail), jail).
Warn("Failed to get ban records for jail")
return []BanRecord{}, nil // Return empty slice instead of error (original behavior)
}
@@ -532,60 +423,29 @@ func (c *RealClient) getBanRecordsInternal(ctx context.Context, jails []string)
// GetLogLines retrieves log lines related to an IP address from the specified jail.
func (c *RealClient) GetLogLines(jail, ip string) ([]string, error) {
return c.GetLogLinesWithLimit(jail, ip, DefaultLogLinesLimit)
return c.GetLogLinesWithLimit(jail, ip, shared.DefaultLogLinesLimit)
}
// GetLogLinesWithLimit returns log lines with configurable limits for memory management.
func (c *RealClient) GetLogLinesWithLimit(jail, ip string, maxLines int) ([]string, error) {
pattern := filepath.Join(c.LogDir, "fail2ban.log*")
files, err := filepath.Glob(pattern)
if err != nil {
return nil, err
}
return c.GetLogLinesWithLimitContext(context.Background(), jail, ip, maxLines)
}
if len(files) == 0 {
// GetLogLinesWithLimitContext returns log lines with configurable limits and context support.
func (c *RealClient) GetLogLinesWithLimitContext(ctx context.Context, jail, ip string, maxLines int) ([]string, error) {
if maxLines == 0 {
return []string{}, nil
}
// Sort files to read in order (current log first, then rotated logs newest to oldest)
sort.Strings(files)
// Use streaming approach with memory limits
config := LogReadConfig{
MaxLines: maxLines,
MaxFileSize: DefaultMaxFileSize,
MaxFileSize: shared.DefaultMaxFileSize,
JailFilter: jail,
IPFilter: ip,
BaseDir: c.LogDir,
}
var allLines []string
totalLines := 0
for _, fpath := range files {
if config.MaxLines > 0 && totalLines >= config.MaxLines {
break
}
// Adjust remaining lines limit
remainingLines := config.MaxLines - totalLines
if remainingLines <= 0 {
break
}
fileConfig := config
fileConfig.MaxLines = remainingLines
lines, err := streamLogFile(fpath, fileConfig)
if err != nil {
getLogger().WithError(err).WithField("file", fpath).Error("Failed to read log file")
continue
}
allLines = append(allLines, lines...)
totalLines += len(lines)
}
return allLines, nil
return collectLogLines(ctx, c.LogDir, config)
}
// ListFilters returns a list of available fail2ban filter files.
@@ -597,8 +457,8 @@ func (c *RealClient) ListFilters() ([]string, error) {
filters := []string{}
for _, entry := range entries {
name := entry.Name()
if strings.HasSuffix(name, ".conf") {
filters = append(filters, strings.TrimSuffix(name, ".conf"))
if strings.HasSuffix(name, shared.ConfExtension) {
filters = append(filters, strings.TrimSuffix(name, shared.ConfExtension))
}
}
return filters, nil
@@ -613,89 +473,86 @@ func (c *RealClient) ListJailsWithContext(ctx context.Context) ([]string, error)
// StatusAllWithContext returns the status of all fail2ban jails with context support.
func (c *RealClient) StatusAllWithContext(ctx context.Context) (string, error) {
globalRunnerManager.mu.RLock()
currentRunner := globalRunnerManager.runner
globalRunnerManager.mu.RUnlock()
currentRunner := GetRunner()
out, err := currentRunner.CombinedOutputWithSudoContext(ctx, c.Path, "status")
out, err := currentRunner.CombinedOutputWithSudoContext(ctx, c.Path, shared.CommandArgStatus)
return string(out), err
}
// StatusJailWithContext returns the status of a specific fail2ban jail with context support.
func (c *RealClient) StatusJailWithContext(ctx context.Context, jail string) (string, error) {
globalRunnerManager.mu.RLock()
currentRunner := globalRunnerManager.runner
globalRunnerManager.mu.RUnlock()
currentRunner := GetRunner()
out, err := currentRunner.CombinedOutputWithSudoContext(ctx, c.Path, "status", jail)
out, err := currentRunner.CombinedOutputWithSudoContext(ctx, c.Path, shared.CommandArgStatus, jail)
return string(out), err
}
// BanIPWithContext bans an IP address in the specified jail with context support.
func (c *RealClient) BanIPWithContext(ctx context.Context, ip, jail string) (int, error) {
if err := CachedValidateIP(ip); err != nil {
if err := CachedValidateIP(ctx, ip); err != nil {
return 0, err
}
if err := CachedValidateJail(jail); err != nil {
if err := CachedValidateJail(ctx, jail); err != nil {
return 0, err
}
globalRunnerManager.mu.RLock()
currentRunner := globalRunnerManager.runner
globalRunnerManager.mu.RUnlock()
currentRunner := GetRunner()
out, err := currentRunner.CombinedOutputWithSudoContext(ctx, c.Path, "set", jail, "banip", ip)
out, err := currentRunner.CombinedOutputWithSudoContext(ctx, c.Path, shared.ActionSet, jail, shared.ActionBanIP, ip)
if err != nil {
return 0, fmt.Errorf("failed to ban IP %s in jail %s: %w", ip, jail, err)
return 0, fmt.Errorf(shared.ErrFailedToBanIP, ip, jail, err)
}
code := strings.TrimSpace(string(out))
if code == Fail2BanStatusSuccess {
if code == shared.Fail2BanStatusSuccess {
return 0, nil
}
if code == Fail2BanStatusAlreadyProcessed {
if code == shared.Fail2BanStatusAlreadyProcessed {
return 1, nil
}
return 0, fmt.Errorf("unexpected output from fail2ban-client: %s", code)
return 0, fmt.Errorf(shared.ErrUnexpectedOutput, code)
}
// UnbanIPWithContext unbans an IP address from the specified jail with context support.
func (c *RealClient) UnbanIPWithContext(ctx context.Context, ip, jail string) (int, error) {
if err := CachedValidateIP(ip); err != nil {
if err := CachedValidateIP(ctx, ip); err != nil {
return 0, err
}
if err := CachedValidateJail(jail); err != nil {
if err := CachedValidateJail(ctx, jail); err != nil {
return 0, err
}
globalRunnerManager.mu.RLock()
currentRunner := globalRunnerManager.runner
globalRunnerManager.mu.RUnlock()
currentRunner := GetRunner()
out, err := currentRunner.CombinedOutputWithSudoContext(ctx, c.Path, "set", jail, "unbanip", ip)
out, err := currentRunner.CombinedOutputWithSudoContext(
ctx,
c.Path,
shared.ActionSet,
jail,
shared.ActionUnbanIP,
ip,
)
if err != nil {
return 0, fmt.Errorf("failed to unban IP %s in jail %s: %w", ip, jail, err)
return 0, fmt.Errorf(shared.ErrFailedToUnbanIP, ip, jail, err)
}
code := strings.TrimSpace(string(out))
if code == Fail2BanStatusSuccess {
if code == shared.Fail2BanStatusSuccess {
return 0, nil
}
if code == Fail2BanStatusAlreadyProcessed {
if code == shared.Fail2BanStatusAlreadyProcessed {
return 1, nil
}
return 0, fmt.Errorf("unexpected output from fail2ban-client: %s", code)
return 0, fmt.Errorf(shared.ErrUnexpectedOutput, code)
}
// BannedInWithContext returns a list of jails where the specified IP address is currently banned with context support.
func (c *RealClient) BannedInWithContext(ctx context.Context, ip string) ([]string, error) {
if err := CachedValidateIP(ip); err != nil {
if err := CachedValidateIP(ctx, ip); err != nil {
return nil, err
}
globalRunnerManager.mu.RLock()
currentRunner := globalRunnerManager.runner
globalRunnerManager.mu.RUnlock()
currentRunner := GetRunner()
out, err := currentRunner.CombinedOutputWithSudoContext(ctx, c.Path, "banned", ip)
out, err := currentRunner.CombinedOutputWithSudoContext(ctx, c.Path, shared.ActionBanned, ip)
if err != nil {
return nil, fmt.Errorf("failed to get banned status for IP %s: %w", ip, err)
}
@@ -709,7 +566,7 @@ func (c *RealClient) GetBanRecordsWithContext(ctx context.Context, jails []strin
// GetLogLinesWithContext retrieves log lines related to an IP address from the specified jail with context support.
func (c *RealClient) GetLogLinesWithContext(ctx context.Context, jail, ip string) ([]string, error) {
return c.GetLogLinesWithLimitAndContext(ctx, jail, ip, DefaultLogLinesLimit)
return c.GetLogLinesWithLimitAndContext(ctx, jail, ip, shared.DefaultLogLinesLimit)
}
// GetLogLinesWithLimitAndContext returns log lines with configurable limits
@@ -719,72 +576,23 @@ func (c *RealClient) GetLogLinesWithLimitAndContext(
jail, ip string,
maxLines int,
) ([]string, error) {
// Check context before starting
select {
case <-ctx.Done():
return nil, ctx.Err()
default:
}
pattern := filepath.Join(c.LogDir, "fail2ban.log*")
files, err := filepath.Glob(pattern)
if err != nil {
if err := ctx.Err(); err != nil {
return nil, err
}
if len(files) == 0 {
if maxLines == 0 {
return []string{}, nil
}
// Sort files to read in order (current log first, then rotated logs newest to oldest)
sort.Strings(files)
// Use streaming approach with memory limits and context support
config := LogReadConfig{
MaxLines: maxLines,
MaxFileSize: DefaultMaxFileSize,
MaxFileSize: shared.DefaultMaxFileSize,
JailFilter: jail,
IPFilter: ip,
BaseDir: c.LogDir,
}
var allLines []string
totalLines := 0
for _, fpath := range files {
// Check context before processing each file
select {
case <-ctx.Done():
return nil, ctx.Err()
default:
}
if config.MaxLines > 0 && totalLines >= config.MaxLines {
break
}
// Adjust remaining lines limit
remainingLines := config.MaxLines - totalLines
if remainingLines <= 0 {
break
}
fileConfig := config
fileConfig.MaxLines = remainingLines
lines, err := streamLogFileWithContext(ctx, fpath, fileConfig)
if err != nil {
if errors.Is(err, ctx.Err()) {
return nil, err // Return context error immediately
}
getLogger().WithError(err).WithField("file", fpath).Error("Failed to read log file")
continue
}
allLines = append(allLines, lines...)
totalLines += len(lines)
}
return allLines, nil
return collectLogLines(ctx, c.LogDir, config)
}
// ListFiltersWithContext returns a list of available fail2ban filter files with context support.
@@ -793,8 +601,8 @@ func (c *RealClient) ListFiltersWithContext(ctx context.Context) ([]string, erro
}
// validateFilterPath validates filter name and returns secure path and log path
func (c *RealClient) validateFilterPath(filter string) (string, string, error) {
if err := CachedValidateFilter(filter); err != nil {
func (c *RealClient) validateFilterPath(ctx context.Context, filter string) (string, string, error) {
if err := CachedValidateFilter(ctx, filter); err != nil {
return "", "", err
}
path := filepath.Join(c.FilterDir, filter+".conf")
@@ -807,7 +615,7 @@ func (c *RealClient) validateFilterPath(filter string) (string, string, error) {
cleanFilterDir, err := filepath.Abs(filepath.Clean(c.FilterDir))
if err != nil {
return "", "", fmt.Errorf("invalid filter directory: %w", err)
return "", "", fmt.Errorf(shared.ErrInvalidFilterDirectory, err)
}
// Ensure the resolved path is within the filter directory
@@ -843,30 +651,18 @@ func (c *RealClient) validateFilterPath(filter string) (string, string, error) {
// TestFilterWithContext tests a fail2ban filter against its configured log files with context support.
func (c *RealClient) TestFilterWithContext(ctx context.Context, filter string) (string, error) {
cleanPath, logPath, err := c.validateFilterPath(filter)
cleanPath, logPath, err := c.validateFilterPath(ctx, filter)
if err != nil {
return "", err
}
globalRunnerManager.mu.RLock()
currentRunner := globalRunnerManager.runner
globalRunnerManager.mu.RUnlock()
currentRunner := GetRunner()
output, err := currentRunner.CombinedOutputWithSudoContext(ctx, Fail2BanRegexCommand, logPath, cleanPath)
output, err := currentRunner.CombinedOutputWithSudoContext(ctx, shared.Fail2BanRegexCommand, logPath, cleanPath)
return string(output), err
}
// TestFilter tests a fail2ban filter against its configured log files and returns the test output.
func (c *RealClient) TestFilter(filter string) (string, error) {
cleanPath, logPath, err := c.validateFilterPath(filter)
if err != nil {
return "", err
}
globalRunnerManager.mu.RLock()
currentRunner := globalRunnerManager.runner
globalRunnerManager.mu.RUnlock()
output, err := currentRunner.CombinedOutputWithSudo(Fail2BanRegexCommand, logPath, cleanPath)
return string(output), err
return c.TestFilterWithContext(context.Background(), filter)
}

89
fail2ban/fail2ban_ban_record_parser_benchmark_test.go
View File

@@ -24,7 +24,10 @@ var benchmarkBanRecordOutput = strings.Join(benchmarkBanRecordData, "\n")
// BenchmarkOriginalBanRecordParsing benchmarks the current implementation
func BenchmarkOriginalBanRecordParsing(b *testing.B) {
parser := NewBanRecordParser()
parser, err := NewBanRecordParser()
if err != nil {
b.Fatal(err)
}
b.ResetTimer()
b.ReportAllocs()
@@ -37,27 +40,15 @@ func BenchmarkOriginalBanRecordParsing(b *testing.B) {
}
}
// BenchmarkOptimizedBanRecordParsing benchmarks the new optimized implementation
func BenchmarkOptimizedBanRecordParsing(b *testing.B) {
parser := NewOptimizedBanRecordParser()
b.ResetTimer()
b.ReportAllocs()
for i := 0; i < b.N; i++ {
_, err := parser.ParseBanRecordsOptimized(benchmarkBanRecordOutput, "sshd")
if err != nil {
b.Fatal(err)
}
}
}
// BenchmarkBanRecordLineParsing compares single line parsing
func BenchmarkBanRecordLineParsing(b *testing.B) {
testLine := "192.168.1.100 2025-07-20 14:30:39 + 2025-07-20 14:40:39 remaining"
b.Run("original", func(b *testing.B) {
parser := NewBanRecordParser()
parser, err := NewBanRecordParser()
if err != nil {
b.Fatal(err)
}
b.ResetTimer()
b.ReportAllocs()
@@ -68,19 +59,6 @@ func BenchmarkBanRecordLineParsing(b *testing.B) {
}
}
})
b.Run("optimized", func(b *testing.B) {
parser := NewOptimizedBanRecordParser()
b.ResetTimer()
b.ReportAllocs()
for i := 0; i < b.N; i++ {
_, err := parser.ParseBanRecordLineOptimized(testLine, "sshd")
if err != nil {
b.Fatal(err)
}
}
})
}
// BenchmarkTimeParsingOptimization compares time parsing implementations
@@ -88,7 +66,11 @@ func BenchmarkTimeParsingOptimization(b *testing.B) {
timeStr := "2025-07-20 14:30:39"
b.Run("original", func(b *testing.B) {
cache := NewTimeParsingCache("2006-01-02 15:04:05")
cache, err := NewTimeParsingCache("2006-01-02 15:04:05")
if err != nil {
b.Fatal(err)
}
b.ResetTimer()
b.ReportAllocs()
@@ -101,7 +83,11 @@ func BenchmarkTimeParsingOptimization(b *testing.B) {
})
b.Run("optimized", func(b *testing.B) {
cache := NewFastTimeCache("2006-01-02 15:04:05")
cache, err := NewFastTimeCache("2006-01-02 15:04:05")
if err != nil {
b.Fatal(err)
}
b.ResetTimer()
b.ReportAllocs()
@@ -120,7 +106,11 @@ func BenchmarkTimeStringBuilding(b *testing.B) {
timeStr := "14:30:39"
b.Run("original", func(b *testing.B) {
cache := NewTimeParsingCache("2006-01-02 15:04:05")
cache, err := NewTimeParsingCache("2006-01-02 15:04:05")
if err != nil {
b.Fatal(err)
}
b.ResetTimer()
b.ReportAllocs()
@@ -130,7 +120,11 @@ func BenchmarkTimeStringBuilding(b *testing.B) {
})
b.Run("optimized", func(b *testing.B) {
cache := NewFastTimeCache("2006-01-02 15:04:05")
cache, err := NewFastTimeCache("2006-01-02 15:04:05")
if err != nil {
b.Fatal(err)
}
b.ResetTimer()
b.ReportAllocs()
@@ -153,8 +147,11 @@ func BenchmarkLargeDataset(b *testing.B) {
}
largeOutput := strings.Join(largeData, "\n")
b.Run("original_large", func(b *testing.B) {
parser := NewBanRecordParser()
b.Run("large_dataset", func(b *testing.B) {
parser, err := NewBanRecordParser()
if err != nil {
b.Fatal(err)
}
b.ResetTimer()
b.ReportAllocs()
@@ -165,19 +162,6 @@ func BenchmarkLargeDataset(b *testing.B) {
}
}
})
b.Run("optimized_large", func(b *testing.B) {
parser := NewOptimizedBanRecordParser()
b.ResetTimer()
b.ReportAllocs()
for i := 0; i < b.N; i++ {
_, err := parser.ParseBanRecordsOptimized(largeOutput, "sshd")
if err != nil {
b.Fatal(err)
}
}
})
}
// BenchmarkDurationFormatting compares duration formatting
@@ -209,7 +193,10 @@ func BenchmarkDurationFormatting(b *testing.B) {
// BenchmarkMemoryPooling tests the effectiveness of object pooling
func BenchmarkMemoryPooling(b *testing.B) {
parser := NewOptimizedBanRecordParser()
parser, err := NewBanRecordParser()
if err != nil {
b.Fatal(err)
}
testLine := "192.168.1.100 2025-07-20 14:30:39 + 2025-07-20 14:40:39 remaining"
b.ResetTimer()
@@ -218,7 +205,7 @@ func BenchmarkMemoryPooling(b *testing.B) {
for i := 0; i < b.N; i++ {
// This should demonstrate reduced allocations due to pooling
for j := 0; j < 10; j++ {
_, err := parser.ParseBanRecordLineOptimized(testLine, "sshd")
_, err := parser.ParseBanRecordLine(testLine, "sshd")
if err != nil {
b.Fatal(err)
}

166
fail2ban/fail2ban_ban_record_parser_compatibility_test.go
View File

@@ -5,55 +5,55 @@ import (
"time"
)
// compareParserResults compares results from original and optimized parsers
func compareParserResults(t *testing.T, originalRecords []BanRecord, originalErr error,
optimizedRecords []BanRecord, optimizedErr error) {
// compareParserResults compares results from two consecutive parser runs
func compareParserResults(t *testing.T, firstRecords []BanRecord, firstErr error,
secondRecords []BanRecord, secondErr error) {
t.Helper()
// Compare errors
if (originalErr == nil) != (optimizedErr == nil) {
t.Fatalf("Error mismatch: original=%v, optimized=%v", originalErr, optimizedErr)
if (firstErr == nil) != (secondErr == nil) {
t.Fatalf("Error mismatch: first=%v, second=%v", firstErr, secondErr)
}
// Compare record counts
if len(originalRecords) != len(optimizedRecords) {
t.Fatalf("Record count mismatch: original=%d, optimized=%d",
len(originalRecords), len(optimizedRecords))
if len(firstRecords) != len(secondRecords) {
t.Fatalf("Record count mismatch: first=%d, second=%d",
len(firstRecords), len(secondRecords))
}
// Compare each record
for i := range originalRecords {
compareRecords(t, i, &originalRecords[i], &optimizedRecords[i])
for i := range firstRecords {
compareRecords(t, i, &firstRecords[i], &secondRecords[i])
}
}
// compareRecords compares individual ban records
func compareRecords(t *testing.T, index int, orig, opt *BanRecord) {
func compareRecords(t *testing.T, index int, first, second *BanRecord) {
t.Helper()
if orig.Jail != opt.Jail {
t.Errorf("Record %d jail mismatch: original=%s, optimized=%s", index, orig.Jail, opt.Jail)
if first.Jail != second.Jail {
t.Errorf("Record %d jail mismatch: first=%s, second=%s", index, first.Jail, second.Jail)
}
if orig.IP != opt.IP {
t.Errorf("Record %d IP mismatch: original=%s, optimized=%s", index, orig.IP, opt.IP)
if first.IP != second.IP {
t.Errorf("Record %d IP mismatch: first=%s, second=%s", index, first.IP, second.IP)
}
// For time comparison, allow small differences due to parsing
if !orig.BannedAt.IsZero() && !opt.BannedAt.IsZero() {
if orig.BannedAt.Unix() != opt.BannedAt.Unix() {
t.Errorf("Record %d banned time mismatch: original=%v, optimized=%v",
index, orig.BannedAt, opt.BannedAt)
if !first.BannedAt.IsZero() && !second.BannedAt.IsZero() {
if first.BannedAt.Unix() != second.BannedAt.Unix() {
t.Errorf("Record %d banned time mismatch: first=%v, second=%v",
index, first.BannedAt, second.BannedAt)
}
}
// Remaining time should be consistent
if orig.Remaining != opt.Remaining {
t.Errorf("Record %d remaining time mismatch: original=%s, optimized=%s",
index, orig.Remaining, opt.Remaining)
if first.Remaining != second.Remaining {
t.Errorf("Record %d remaining time mismatch: first=%s, second=%s",
index, first.Remaining, second.Remaining)
}
}
// TestParserCompatibility ensures the optimized parser produces identical results to the original
func TestParserCompatibility(t *testing.T) {
// TestParserDeterminism ensures the parser produces identical results across consecutive runs
func TestParserDeterminism(t *testing.T) {
testCases := []struct {
name string
input string
@@ -97,68 +97,76 @@ func TestParserCompatibility(t *testing.T) {
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
// Parse with original parser
originalParser := NewBanRecordParser()
originalRecords, originalErr := originalParser.ParseBanRecords(tc.input, tc.jail)
// Validates parser determinism by running twice with identical input
parser1, err := NewBanRecordParser()
if err != nil {
t.Fatal(err)
}
// Parse with optimized parser
optimizedParser := NewOptimizedBanRecordParser()
optimizedRecords, optimizedErr := optimizedParser.ParseBanRecordsOptimized(tc.input, tc.jail)
// First parse
firstRecords, firstErr := parser1.ParseBanRecords(tc.input, tc.jail)
compareParserResults(t, originalRecords, originalErr, optimizedRecords, optimizedErr)
// Second parse with fresh parser (should produce identical results)
parser2, err := NewBanRecordParser()
if err != nil {
t.Fatal(err)
}
secondRecords, secondErr := parser2.ParseBanRecords(tc.input, tc.jail)
compareParserResults(t, firstRecords, firstErr, secondRecords, secondErr)
})
}
}
// compareSingleRecords compares individual parsed records
func compareSingleRecords(t *testing.T, originalRecord *BanRecord, originalErr error,
optimizedRecord *BanRecord, optimizedErr error) {
func compareSingleRecords(t *testing.T, firstRecord *BanRecord, firstErr error,
secondRecord *BanRecord, secondErr error) {
t.Helper()
// Compare errors
if (originalErr == nil) != (optimizedErr == nil) {
t.Fatalf("Error mismatch: original=%v, optimized=%v", originalErr, optimizedErr)
if (firstErr == nil) != (secondErr == nil) {
t.Fatalf("Error mismatch: first=%v, second=%v", firstErr, secondErr)
}
// If both have errors, that's fine - they should be the same type
if originalErr != nil && optimizedErr != nil {
if firstErr != nil && secondErr != nil {
return
}
// Compare records
if (originalRecord == nil) != (optimizedRecord == nil) {
t.Fatalf("Record nil mismatch: original=%v, optimized=%v",
originalRecord == nil, optimizedRecord == nil)
if (firstRecord == nil) != (secondRecord == nil) {
t.Fatalf("Record nil mismatch: first=%v, second=%v",
firstRecord == nil, secondRecord == nil)
}
if originalRecord != nil && optimizedRecord != nil {
compareRecordFields(t, originalRecord, optimizedRecord)
if firstRecord != nil && secondRecord != nil {
compareRecordFields(t, firstRecord, secondRecord)
}
}
// compareRecordFields compares fields of two ban records
func compareRecordFields(t *testing.T, original, optimized *BanRecord) {
func compareRecordFields(t *testing.T, first, second *BanRecord) {
t.Helper()
if original.Jail != optimized.Jail {
t.Errorf("Jail mismatch: original=%s, optimized=%s",
original.Jail, optimized.Jail)
if first.Jail != second.Jail {
t.Errorf("Jail mismatch: first=%s, second=%s",
first.Jail, second.Jail)
}
if original.IP != optimized.IP {
t.Errorf("IP mismatch: original=%s, optimized=%s",
original.IP, optimized.IP)
if first.IP != second.IP {
t.Errorf("IP mismatch: first=%s, second=%s",
first.IP, second.IP)
}
// Time comparison with tolerance
if !original.BannedAt.IsZero() && !optimized.BannedAt.IsZero() {
if original.BannedAt.Unix() != optimized.BannedAt.Unix() {
t.Errorf("BannedAt mismatch: original=%v, optimized=%v",
original.BannedAt, optimized.BannedAt)
if !first.BannedAt.IsZero() && !second.BannedAt.IsZero() {
if first.BannedAt.Unix() != second.BannedAt.Unix() {
t.Errorf("BannedAt mismatch: first=%v, second=%v",
first.BannedAt, second.BannedAt)
}
}
}
// TestParserCompatibilityLineByLine tests individual line parsing compatibility
func TestParserCompatibilityLineByLine(t *testing.T) {
// TestParserDeterminismLineByLine tests individual line parsing determinism
func TestParserDeterminismLineByLine(t *testing.T) {
testLines := []struct {
name string
line string
@@ -193,22 +201,33 @@ func TestParserCompatibilityLineByLine(t *testing.T) {
for _, tc := range testLines {
t.Run(tc.name, func(t *testing.T) {
// Parse with original parser
originalParser := NewBanRecordParser()
originalRecord, originalErr := originalParser.ParseBanRecordLine(tc.line, tc.jail)
// Validates parser determinism by running twice with identical input
parser1, err := NewBanRecordParser()
if err != nil {
t.Fatal(err)
}
// Parse with optimized parser
optimizedParser := NewOptimizedBanRecordParser()
optimizedRecord, optimizedErr := optimizedParser.ParseBanRecordLineOptimized(tc.line, tc.jail)
// First parse
firstRecord, firstErr := parser1.ParseBanRecordLine(tc.line, tc.jail)
compareSingleRecords(t, originalRecord, originalErr, optimizedRecord, optimizedErr)
// Second parse with fresh parser (should produce identical results)
parser2, err := NewBanRecordParser()
if err != nil {
t.Fatal(err)
}
secondRecord, secondErr := parser2.ParseBanRecordLine(tc.line, tc.jail)
compareSingleRecords(t, firstRecord, firstErr, secondRecord, secondErr)
})
}
}
// TestOptimizedParserStatistics tests the statistics functionality
func TestOptimizedParserStatistics(t *testing.T) {
parser := NewOptimizedBanRecordParser()
// TestParserStatistics tests the statistics functionality
func TestParserStatistics(t *testing.T) {
parser, err := NewBanRecordParser()
if err != nil {
t.Fatal(err)
}
// Initial stats should be zero
parseCount, errorCount := parser.GetStats()
@@ -221,7 +240,7 @@ func TestOptimizedParserStatistics(t *testing.T) {
10.0.0.50 2025-07-20 14:36:59 + 2025-07-20 14:46:59 remaining`
records, err := parser.ParseBanRecordsOptimized(input, "sshd")
records, err := parser.ParseBanRecords(input, "sshd")
if err != nil {
t.Fatalf("Unexpected error: %v", err)
}
@@ -242,7 +261,10 @@ func TestOptimizedParserStatistics(t *testing.T) {
// TestTimeParsingOptimizations tests the optimized time parsing
func TestTimeParsingOptimizations(t *testing.T) {
cache := NewFastTimeCache("2006-01-02 15:04:05")
cache, err := NewFastTimeCache("2006-01-02 15:04:05")
if err != nil {
t.Fatal(err)
}
testTimeStr := "2025-07-20 14:30:39"
@@ -270,7 +292,10 @@ func TestTimeParsingOptimizations(t *testing.T) {
// TestStringBuildingOptimizations tests the optimized string building
func TestStringBuildingOptimizations(t *testing.T) {
cache := NewFastTimeCache("2006-01-02 15:04:05")
cache, err := NewFastTimeCache("2006-01-02 15:04:05")
if err != nil {
t.Fatal(err)
}
dateStr := "2025-07-20"
timeStr := "14:30:39"
@@ -284,14 +309,17 @@ func TestStringBuildingOptimizations(t *testing.T) {
// BenchmarkParserStatistics tests performance impact of statistics tracking
func BenchmarkParserStatistics(b *testing.B) {
parser := NewOptimizedBanRecordParser()
parser, err := NewBanRecordParser()
if err != nil {
b.Fatal(err)
}
testLine := "192.168.1.100 2025-07-20 14:30:39 + 2025-07-20 14:40:39 remaining"
b.ResetTimer()
b.ReportAllocs()
for i := 0; i < b.N; i++ {
_, err := parser.ParseBanRecordLineOptimized(testLine, "sshd")
_, err := parser.ParseBanRecordLine(testLine, "sshd")
if err != nil {
b.Fatal(err)
}

56
fail2ban/fail2ban_ban_record_parser_test.go
View File

@@ -8,7 +8,10 @@ import (
)
func TestBanRecordParser(t *testing.T) {
parser := NewBanRecordParser()
parser, err := NewBanRecordParser()
if err != nil {
t.Fatal(err)
}
tests := []struct {
name string
@@ -77,9 +80,7 @@ func TestBanRecordParser(t *testing.T) {
if record == nil {
t.Fatal("Expected record, got nil")
}
if record.IP != tt.wantIP {
} else if record.IP != tt.wantIP {
t.Errorf("IP mismatch: got %s, want %s", record.IP, tt.wantIP)
}
@@ -91,7 +92,10 @@ func TestBanRecordParser(t *testing.T) {
}
func TestParseBanRecords(t *testing.T) {
parser := NewBanRecordParser()
parser, err := NewBanRecordParser()
if err != nil {
t.Fatal(err)
}
output := strings.Join([]string{
"192.168.1.100 2023-12-01 14:30:45 + 2023-12-02 14:30:45 remaining",
@@ -106,10 +110,10 @@ func TestParseBanRecords(t *testing.T) {
t.Fatalf("ParseBanRecords failed: %v", err)
}
expectedIPs := []string{"192.168.1.100", "192.168.1.101", "invalid", "192.168.1.102"}
// Note: empty line is skipped, but "invalid" is treated as simple format
if len(records) != 4 {
t.Fatalf("Expected 4 records (empty line skipped), got %d", len(records))
expectedIPs := []string{"192.168.1.100", "192.168.1.101", "192.168.1.102"}
// Note: empty line and invalid IP are both skipped due to validation
if len(records) != 3 {
t.Fatalf("Expected 3 records (empty line and invalid IP skipped), got %d", len(records))
}
for i, record := range records {
@@ -132,9 +136,7 @@ func TestParseBanRecordLineOptimized(t *testing.T) {
if record == nil {
t.Fatal("Expected record, got nil")
}
if record.IP != "192.168.1.100" {
} else if record.IP != "192.168.1.100" {
t.Errorf("IP mismatch: got %s, want 192.168.1.100", record.IP)
}
@@ -158,7 +160,10 @@ func TestParseBanRecordsOptimized(t *testing.T) {
}
func BenchmarkParseBanRecordLine(b *testing.B) {
parser := NewBanRecordParser()
parser, err := NewBanRecordParser()
if err != nil {
b.Fatal(err)
}
line := "192.168.1.100 2023-12-01 14:30:45 + 2023-12-02 14:30:45 remaining"
b.ResetTimer()
@@ -168,7 +173,10 @@ func BenchmarkParseBanRecordLine(b *testing.B) {
}
func BenchmarkParseBanRecords(b *testing.B) {
parser := NewBanRecordParser()
parser, err := NewBanRecordParser()
if err != nil {
b.Fatal(err)
}
output := strings.Repeat("192.168.1.100 2023-12-01 14:30:45 + 2023-12-02 14:30:45 remaining\n", 100)
b.ResetTimer()
@@ -179,7 +187,10 @@ func BenchmarkParseBanRecords(b *testing.B) {
// Test error handling for invalid time formats
func TestParseBanRecordInvalidTime(t *testing.T) {
parser := NewBanRecordParser()
parser, err := NewBanRecordParser()
if err != nil {
t.Fatal(err)
}
// Invalid ban time should be skipped (original behavior) - must have 8+ fields
line := "192.168.1.100 invalid-date 14:30:45 + 2023-12-02 14:30:45 remaining extra"
@@ -201,7 +212,10 @@ func TestParseBanRecordInvalidTime(t *testing.T) {
// Test concurrent access to parser
func TestBanRecordParserConcurrent(t *testing.T) {
parser := NewBanRecordParser()
parser, err := NewBanRecordParser()
if err != nil {
t.Fatal(err)
}
line := "192.168.1.100 2023-12-01 14:30:45 + 2023-12-02 14:30:45 remaining"
const numGoroutines = 10
@@ -231,7 +245,10 @@ func TestBanRecordParserConcurrent(t *testing.T) {
// TestRealWorldBanRecordPatterns tests with actual patterns from production logs
func TestRealWorldBanRecordPatterns(t *testing.T) {
parser := NewBanRecordParser()
parser, err := NewBanRecordParser()
if err != nil {
t.Fatal(err)
}
// Real patterns observed in production fail2ban
realWorldPatterns := []struct {
@@ -309,7 +326,10 @@ func TestRealWorldBanRecordPatterns(t *testing.T) {
// TestProductionLogTimingPatterns verifies timing patterns from real logs
func TestProductionLogTimingPatterns(t *testing.T) {
parser := NewBanRecordParser()
parser, err := NewBanRecordParser()
if err != nil {
t.Fatal(err)
}
// Test various real production patterns
tests := []struct {

27
fail2ban/fail2ban_error_handling_fix_test.go
View File

@@ -1,6 +1,7 @@
package fail2ban
import (
"context"
"os"
"path/filepath"
"strings"
@@ -17,7 +18,7 @@ func TestGetLogLinesErrorHandling(t *testing.T) {
// Set log directory to non-existent path
SetLogDir("/nonexistent/path/that/should/not/exist")
lines, err := GetLogLines("sshd", "")
lines, err := GetLogLines(context.Background(), "sshd", "")
if err != nil {
t.Logf("Correctly handled non-existent log directory: %v", err)
}
@@ -36,7 +37,7 @@ func TestGetLogLinesErrorHandling(t *testing.T) {
SetLogDir(tempDir)
lines, err := GetLogLines("sshd", "192.168.1.100")
lines, err := GetLogLines(context.Background(), "sshd", "192.168.1.100")
if err != nil {
t.Errorf("Should not error on empty directory, got: %v", err)
}
@@ -65,7 +66,7 @@ func TestGetLogLinesErrorHandling(t *testing.T) {
}
// Test filtering by jail
lines, err := GetLogLines("sshd", "")
lines, err := GetLogLines(context.Background(), "sshd", "")
if err != nil {
t.Errorf("GetLogLines should not error with valid log: %v", err)
}
@@ -101,7 +102,7 @@ func TestGetLogLinesErrorHandling(t *testing.T) {
}
// Test filtering by IP
lines, err := GetLogLines("", "192.168.1.100")
lines, err := GetLogLines(context.Background(), "", "192.168.1.100")
if err != nil {
t.Errorf("GetLogLines should not error with valid log: %v", err)
}
@@ -138,7 +139,7 @@ func TestGetLogLinesWithLimitErrorHandling(t *testing.T) {
}
// Test with zero limit
lines, err := GetLogLinesWithLimit("sshd", "", 0)
lines, err := GetLogLinesWithLimit(context.Background(), "sshd", "", 0)
if err != nil {
t.Errorf("GetLogLinesWithLimit should not error with zero limit: %v", err)
}
@@ -163,15 +164,15 @@ func TestGetLogLinesWithLimitErrorHandling(t *testing.T) {
t.Fatalf("Failed to create test log file: %v", err)
}
// Test with negative limit (should be treated as unlimited)
lines, err := GetLogLinesWithLimit("sshd", "", -1)
if err != nil {
t.Errorf("GetLogLinesWithLimit should not error with negative limit: %v", err)
// Test with negative limit (should be rejected with validation error)
_, err = GetLogLinesWithLimit(context.Background(), "sshd", "", -1)
if err == nil {
t.Error("GetLogLinesWithLimit should error with negative limit")
}
// Should return available lines
if len(lines) == 0 {
t.Error("Expected lines with negative limit (unlimited)")
// Error should indicate validation failure
if !strings.Contains(err.Error(), "must be non-negative") {
t.Errorf("Expected validation error for negative limit, got: %v", err)
}
})
@@ -194,7 +195,7 @@ func TestGetLogLinesWithLimitErrorHandling(t *testing.T) {
}
// Test with limit of 2
lines, err := GetLogLinesWithLimit("sshd", "", 2)
lines, err := GetLogLinesWithLimit(context.Background(), "sshd", "", 2)
if err != nil {
t.Errorf("GetLogLinesWithLimit should not error: %v", err)
}

249
fail2ban/fail2ban_fail2ban_test.go
View File

@@ -1,82 +1,18 @@
package fail2ban
import (
"context"
"fmt"
"os"
"path/filepath"
"reflect"
"strings"
"testing"
"time"
"github.com/ivuorinen/f2b/shared"
)
func TestNewClient(t *testing.T) {
tests := []struct {
name string
hasPrivileges bool
expectError bool
errorContains string
}{
{
name: "with sudo privileges",
hasPrivileges: true,
expectError: false,
},
{
name: "without sudo privileges",
hasPrivileges: false,
expectError: true,
errorContains: "fail2ban operations require sudo privileges",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Set environment variable to force sudo checking in tests
t.Setenv("F2B_TEST_SUDO", "true")
// Set up mock environment
_, cleanup := SetupMockEnvironmentWithSudo(t, tt.hasPrivileges)
defer cleanup()
// Get the mock runner that was set up
mockRunner := GetRunner().(*MockRunner)
if tt.hasPrivileges {
mockRunner.SetResponse("fail2ban-client -V", []byte("0.11.2"))
mockRunner.SetResponse("sudo fail2ban-client -V", []byte("0.11.2"))
mockRunner.SetResponse("fail2ban-client ping", []byte("pong"))
mockRunner.SetResponse("sudo fail2ban-client ping", []byte("pong"))
mockRunner.SetResponse(
"fail2ban-client status",
[]byte("Status\n|- Number of jail: 1\n`- Jail list: sshd"),
)
mockRunner.SetResponse(
"sudo fail2ban-client status",
[]byte("Status\n|- Number of jail: 1\n`- Jail list: sshd"),
)
} else {
// For unprivileged tests, set up basic responses for non-sudo commands
mockRunner.SetResponse("fail2ban-client -V", []byte("0.11.2"))
mockRunner.SetResponse("fail2ban-client ping", []byte("pong"))
mockRunner.SetResponse("fail2ban-client status", []byte("Status\n|- Number of jail: 1\n`- Jail list: sshd"))
}
client, err := NewClient(DefaultLogDir, DefaultFilterDir)
AssertError(t, err, tt.expectError, tt.name)
if tt.expectError {
if tt.errorContains != "" && err != nil && !strings.Contains(err.Error(), tt.errorContains) {
t.Errorf("expected error to contain %q, got %q", tt.errorContains, err.Error())
}
return
}
if client == nil {
t.Fatal("expected client to be non-nil")
}
})
}
}
func TestListJails(t *testing.T) {
tests := []struct {
name string
@@ -128,12 +64,12 @@ func TestListJails(t *testing.T) {
if tt.expectError {
// For error cases, we expect NewClient to fail
_, err := NewClient(DefaultLogDir, DefaultFilterDir)
_, err := NewClient(shared.DefaultLogDir, shared.DefaultFilterDir)
AssertError(t, err, true, tt.name)
return
}
client, err := NewClient(DefaultLogDir, DefaultFilterDir)
client, err := NewClient(shared.DefaultLogDir, shared.DefaultFilterDir)
AssertError(t, err, false, "create client")
jails, err := client.ListJails()
@@ -163,7 +99,7 @@ func TestStatusAll(t *testing.T) {
mock.SetResponse("fail2ban-client status", []byte(expectedOutput))
mock.SetResponse("sudo fail2ban-client status", []byte(expectedOutput))
client, err := NewClient(DefaultLogDir, DefaultFilterDir)
client, err := NewClient(shared.DefaultLogDir, shared.DefaultFilterDir)
AssertError(t, err, false, "create client")
output, err := client.StatusAll()
@@ -186,7 +122,7 @@ func TestStatusJail(t *testing.T) {
mock.SetResponse("fail2ban-client status sshd", []byte(expectedOutput))
mock.SetResponse("sudo fail2ban-client status sshd", []byte(expectedOutput))
client, err := NewClient(DefaultLogDir, DefaultFilterDir)
client, err := NewClient(shared.DefaultLogDir, shared.DefaultFilterDir)
AssertError(t, err, false, "create client")
output, err := client.StatusJail("sshd")
@@ -249,7 +185,7 @@ func TestBanIP(t *testing.T) {
mock.SetResponse(fmt.Sprintf("sudo fail2ban-client set %s banip %s", tt.jail, tt.ip), []byte(tt.mockResponse))
}
client, err := NewClient(DefaultLogDir, DefaultFilterDir)
client, err := NewClient(shared.DefaultLogDir, shared.DefaultFilterDir)
AssertError(t, err, false, "create client")
code, err := client.BanIP(tt.ip, tt.jail)
@@ -306,7 +242,7 @@ func TestUnbanIP(t *testing.T) {
[]byte(tt.mockResponse),
)
client, err := NewClient(DefaultLogDir, DefaultFilterDir)
client, err := NewClient(shared.DefaultLogDir, shared.DefaultFilterDir)
AssertError(t, err, false, "create client")
code, err := client.UnbanIP(tt.ip, tt.jail)
@@ -372,7 +308,7 @@ func TestBannedIn(t *testing.T) {
mock.SetResponse(fmt.Sprintf("fail2ban-client banned %s", tt.ip), []byte(tt.mockResponse))
mock.SetResponse(fmt.Sprintf("sudo fail2ban-client banned %s", tt.ip), []byte(tt.mockResponse))
client, err := NewClient(DefaultLogDir, DefaultFilterDir)
client, err := NewClient(shared.DefaultLogDir, shared.DefaultFilterDir)
AssertError(t, err, false, "create client")
jails, err := client.BannedIn(tt.ip)
@@ -410,7 +346,7 @@ func TestGetBanRecords(t *testing.T) {
unbanTime.Format("2006-01-02 15:04:05"))
mock.SetResponse("sudo fail2ban-client get sshd banip --with-time", []byte(mockBanOutput))
client, err := NewClient(DefaultLogDir, DefaultFilterDir)
client, err := NewClient(shared.DefaultLogDir, shared.DefaultFilterDir)
AssertError(t, err, false, "create client")
records, err := client.GetBanRecords([]string{"sshd"})
@@ -447,9 +383,7 @@ func TestGetLogLines(t *testing.T) {
}
mock := NewMockRunner()
mock.SetResponse("fail2ban-client -V", []byte("0.11.2"))
mock.SetResponse("fail2ban-client ping", []byte("pong"))
mock.SetResponse("fail2ban-client status", []byte("Status\n|- Number of jail: 1\n`- Jail list: sshd"))
StandardMockSetup(mock)
SetRunner(mock)
tests := []struct {
@@ -486,7 +420,7 @@ func TestGetLogLines(t *testing.T) {
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
lines, err := GetLogLines(tt.jail, tt.ip)
lines, err := GetLogLines(context.Background(), tt.jail, tt.ip)
AssertError(t, err, false, "get log lines")
if len(lines) != tt.expectedLines {
@@ -495,6 +429,47 @@ func TestGetLogLines(t *testing.T) {
})
}
}
func TestGetLogLinesWithLimitPrefersRecent(t *testing.T) {
originalDir := GetLogDir()
SetLogDir(t.TempDir())
defer SetLogDir(originalDir)
logDir := GetLogDir()
oldPath := filepath.Join(logDir, "fail2ban.log.1")
newPath := filepath.Join(logDir, "fail2ban.log")
// Older rotated log with more entries than the requested limit
oldContent := "old-entry-1\nold-entry-2\nold-entry-3\n"
if err := os.WriteFile(oldPath, []byte(oldContent), 0o600); err != nil {
t.Fatalf("failed to create rotated log: %v", err)
}
// Current log with the most recent entries
newContent := "new-entry-1\nnew-entry-2\n"
if err := os.WriteFile(newPath, []byte(newContent), 0o600); err != nil {
t.Fatalf("failed to create current log: %v", err)
}
lines, err := GetLogLinesWithLimit(context.Background(), "", "", 2)
if err != nil {
t.Fatalf("GetLogLinesWithLimit returned error: %v", err)
}
expected := []string{"new-entry-1", "new-entry-2"}
if !reflect.DeepEqual(lines, expected) {
t.Fatalf("expected %v, got %v", expected, lines)
}
client := &RealClient{LogDir: logDir}
clientLines, err := client.GetLogLinesWithLimit("", "", 2)
if err != nil {
t.Fatalf("RealClient.GetLogLinesWithLimit returned error: %v", err)
}
if !reflect.DeepEqual(clientLines, expected) {
t.Fatalf("client expected %v, got %v", expected, clientLines)
}
}
func TestListFilters(t *testing.T) {
// Set ALLOW_DEV_PATHS for test to use temp directory
@@ -525,7 +500,7 @@ func TestListFilters(t *testing.T) {
SetRunner(mock)
// Create client with the temporary filter directory
client, err := NewClient(DefaultLogDir, filterDir)
client, err := NewClient(shared.DefaultLogDir, filterDir)
AssertError(t, err, false, "create client")
// Test ListFilters with the temporary directory
@@ -581,7 +556,7 @@ logpath = /var/log/auth.log`
mock.SetResponse("sudo fail2ban-regex /var/log/auth.log "+filterPath, []byte(expectedOutput))
// Create client with the temp directory as the filter directory
client, err := NewClient(DefaultLogDir, tempDir)
client, err := NewClient(shared.DefaultLogDir, tempDir)
AssertError(t, err, false, "create client")
// Test the actual created filter
@@ -600,52 +575,114 @@ logpath = /var/log/auth.log`
}
func TestVersionComparison(t *testing.T) {
// This tests the version comparison logic indirectly through NewClient
tests := []struct {
name string
version string
expectError bool
name string
versionOutput string
expectError bool
errorSubstring string
}{
{
name: "version 0.11.2 should work",
version: "0.11.2",
expectError: false,
name: "prefixed supported version",
versionOutput: "Fail2Ban v0.11.2",
expectError: false,
},
{
name: "version 0.12.0 should work",
version: "0.12.0",
expectError: false,
name: "plain supported version",
versionOutput: "0.12.0",
expectError: false,
},
{
name: "version 0.10.9 should fail",
version: "0.10.9",
expectError: true,
name: "unsupported version",
versionOutput: "Fail2Ban v0.10.9",
expectError: true,
errorSubstring: "fail2ban >=0.11.0 required",
},
{
name: "unparseable version",
versionOutput: "unexpected output",
expectError: true,
errorSubstring: "failed to parse fail2ban version",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Set up mock environment with privileges based on expected outcome
_, cleanup := SetupMockEnvironmentWithSudo(t, !tt.expectError)
_, cleanup := SetupMockEnvironmentWithSudo(t, true)
defer cleanup()
// Configure specific responses for this test
mock := GetRunner().(*MockRunner)
mock.SetResponse("fail2ban-client -V", []byte(tt.version))
mock.SetResponse("sudo fail2ban-client -V", []byte(tt.version))
mock.SetResponse("fail2ban-client -V", []byte(tt.versionOutput))
mock.SetResponse("sudo fail2ban-client -V", []byte(tt.versionOutput))
if !tt.expectError {
mock.SetResponse("fail2ban-client ping", []byte("pong"))
mock.SetResponse("sudo fail2ban-client ping", []byte("pong"))
mock.SetResponse("fail2ban-client status", []byte("Status\n|- Number of jail: 1\n`- Jail list: sshd"))
mock.SetResponse(
"sudo fail2ban-client status",
[]byte("Status\n|- Number of jail: 1\n`- Jail list: sshd"),
)
statusOutput := []byte("Status\n|- Number of jail: 1\n`- Jail list: sshd")
mock.SetResponse("fail2ban-client status", statusOutput)
mock.SetResponse("sudo fail2ban-client status", statusOutput)
}
_, err := NewClient(DefaultLogDir, DefaultFilterDir)
_, err := NewClient(shared.DefaultLogDir, shared.DefaultFilterDir)
AssertError(t, err, tt.expectError, tt.name)
if tt.expectError && tt.errorSubstring != "" {
if err == nil || !strings.Contains(err.Error(), tt.errorSubstring) {
t.Fatalf("expected error containing %q, got %v", tt.errorSubstring, err)
}
}
})
}
}
func TestExtractFail2BanVersion(t *testing.T) {
tests := []struct {
name string
input string
expect string
expectErr bool
}{
{
name: "prefixed output",
input: "Fail2Ban v0.11.2",
expect: "0.11.2",
},
{
name: "with extra context",
input: "fail2ban 0.12.0 (Python 3)",
expect: "0.12.0",
},
{
name: "plain version",
input: "0.13.1",
expect: "0.13.1",
},
{
name: "leading v",
input: "v1.0.0",
expect: "1.0.0",
},
{
name: "invalid output",
input: "not a version",
expectErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
version, err := ExtractFail2BanVersion(tt.input)
if tt.expectErr {
if err == nil {
t.Fatalf("expected error for input %q", tt.input)
}
return
}
if err != nil {
t.Fatalf("unexpected error for input %q: %v", tt.input, err)
}
if version != tt.expect {
t.Fatalf("expected version %q, got %q", tt.expect, version)
}
})
}
}

134
fail2ban/fail2ban_integration_sudo_test.go
View File

@@ -3,40 +3,14 @@ package fail2ban
import (
"strings"
"testing"
"github.com/ivuorinen/f2b/shared"
)
// setupMockRunnerForPrivilegedTest configures mock responses for privileged tests
func setupMockRunnerForPrivilegedTest(mockRunner *MockRunner) {
// Set up responses for successful client creation
mockRunner.SetResponse("fail2ban-client -V", []byte("0.11.2"))
mockRunner.SetResponse("sudo fail2ban-client -V", []byte("0.11.2"))
mockRunner.SetResponse("fail2ban-client ping", []byte("pong"))
mockRunner.SetResponse("sudo fail2ban-client ping", []byte("pong"))
mockRunner.SetResponse(
"fail2ban-client status",
[]byte("Status\n|- Number of jail: 1\n`- Jail list: sshd"),
)
mockRunner.SetResponse(
"sudo fail2ban-client status",
[]byte("Status\n|- Number of jail: 1\n`- Jail list: sshd"),
)
// Set up responses for operations (both sudo and non-sudo for root users)
mockRunner.SetResponse("sudo fail2ban-client set sshd banip 192.168.1.100", []byte("0"))
mockRunner.SetResponse("fail2ban-client set sshd banip 192.168.1.100", []byte("0"))
mockRunner.SetResponse("sudo fail2ban-client set sshd unbanip 192.168.1.100", []byte("0"))
mockRunner.SetResponse("fail2ban-client set sshd unbanip 192.168.1.100", []byte("0"))
mockRunner.SetResponse("sudo fail2ban-client banned 192.168.1.100", []byte(`["sshd"]`))
mockRunner.SetResponse("fail2ban-client banned 192.168.1.100", []byte(`["sshd"]`))
}
// setupMockRunnerForUnprivilegedTest configures mock responses for unprivileged tests
func setupMockRunnerForUnprivilegedTest(mockRunner *MockRunner) {
// For unprivileged tests, set up basic responses for non-sudo commands
mockRunner.SetResponse("fail2ban-client -V", []byte("0.11.2"))
mockRunner.SetResponse("fail2ban-client ping", []byte("pong"))
mockRunner.SetResponse("fail2ban-client status", []byte("Status\n|- Number of jail: 1\n`- Jail list: sshd"))
mockRunner.SetResponse("fail2ban-client banned 192.168.1.100", []byte(`[]`))
// Use standard mock setup as the base
StandardMockSetup(mockRunner)
}
// testClientOperations tests various client operations
@@ -84,45 +58,62 @@ func testClientOperations(t *testing.T, client Client, expectOperationErr bool)
// TestSudoIntegrationWithClient tests the full integration of sudo checking with client operations
func TestSudoIntegrationWithClient(t *testing.T) {
// Test normal client creation (in test environment, sudo checking is skipped)
t.Run("normal client creation", func(t *testing.T) {
// Modern standardized setup with automatic cleanup
_, cleanup := SetupMockEnvironmentWithSudo(t, true)
defer cleanup()
// Get the mock runner and configure additional responses
mockRunner := GetRunner().(*MockRunner)
setupMockRunnerForPrivilegedTest(mockRunner)
// Test client creation
client, err := NewClient(shared.DefaultLogDir, shared.DefaultFilterDir)
if err != nil {
t.Fatalf("unexpected client creation error: %v", err)
}
if client == nil {
t.Fatal("expected non-nil client")
}
testClientOperations(t, client, false)
})
}
func TestSudoRequirementsIntegration(t *testing.T) {
tests := []struct {
name string
hasPrivileges bool
isRoot bool
expectClientError bool
expectOperationErr bool
description string
name string
hasPrivileges bool
isRoot bool
expectError bool
description string
}{
{
name: "root user can perform all operations",
hasPrivileges: true,
isRoot: true,
expectClientError: false,
expectOperationErr: false,
description: "root user should be able to create client and perform operations",
name: "root user has privileges",
hasPrivileges: true,
isRoot: true,
expectError: false,
description: "root user should pass sudo requirements check",
},
{
name: "user with sudo privileges can perform operations",
hasPrivileges: true,
isRoot: false,
expectClientError: false,
expectOperationErr: false,
description: "user in sudo group should be able to create client and perform operations",
name: "user with sudo privileges passes",
hasPrivileges: true,
isRoot: false,
expectError: false,
description: "user in sudo group should pass sudo requirements check",
},
{
name: "regular user cannot create client",
hasPrivileges: false,
isRoot: false,
expectClientError: true,
expectOperationErr: true,
description: "regular user should fail at client creation",
name: "regular user fails sudo check",
hasPrivileges: false,
isRoot: false,
expectError: true,
description: "regular user should fail sudo requirements check",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Set environment variable to force sudo checking in tests
t.Setenv("F2B_TEST_SUDO", "true")
// Modern standardized setup with automatic cleanup
_, cleanup := SetupMockEnvironmentWithSudo(t, tt.hasPrivileges)
defer cleanup()
@@ -135,20 +126,12 @@ func TestSudoIntegrationWithClient(t *testing.T) {
mockChecker.MockHasPrivileges = true
}
// Get the mock runner and configure additional responses
mockRunner := GetRunner().(*MockRunner)
if tt.hasPrivileges {
setupMockRunnerForPrivilegedTest(mockRunner)
} else {
setupMockRunnerForUnprivilegedTest(mockRunner)
}
// Test sudo requirements directly
err := CheckSudoRequirements()
// Test client creation
client, err := NewClient(DefaultLogDir, DefaultFilterDir)
if tt.expectClientError {
if tt.expectError {
if err == nil {
t.Fatal("expected client creation to fail")
t.Fatal("expected sudo requirements check to fail")
}
if !strings.Contains(err.Error(), "fail2ban operations require sudo privileges") {
t.Errorf("expected sudo privilege error, got: %v", err)
@@ -157,14 +140,8 @@ func TestSudoIntegrationWithClient(t *testing.T) {
}
if err != nil {
t.Fatalf("unexpected client creation error: %v", err)
t.Fatalf("unexpected sudo requirements error: %v", err)
}
if client == nil {
t.Fatal("expected non-nil client")
}
testClientOperations(t, client, tt.expectOperationErr)
})
}
}
@@ -381,11 +358,8 @@ func TestSudoWithDifferentCommands(t *testing.T) {
t.Errorf("RequiresSudo(%s, %v) = %v, want %v", tt.command, tt.args, requiresSudo, tt.expectsSudo)
}
// Reset to clean mock environment for this test iteration
_, cleanup := SetupMockEnvironment(t)
defer cleanup()
// Configure the mock runner with expected response
// Note: Reusing outer mock environment to avoid nested cleanup issues
mockRunner := GetRunner().(*MockRunner)
expectedCall := tt.expectedPrefix + " " + strings.Join(tt.args, " ")
mockRunner.SetResponse(expectedCall, []byte("mock response"))

335
fail2ban/fail2ban_log_performance_benchmark_test.go
View File

@@ -1,19 +1,15 @@
package fail2ban
import (
"fmt"
"context"
"os"
"path/filepath"
"strings"
"testing"
)
// BenchmarkOriginalLogParsing benchmarks the current log parsing implementation
func BenchmarkOriginalLogParsing(b *testing.B) {
// Set up test environment with test data
testLogFile := filepath.Join("testdata", "fail2ban_full.log")
// Ensure test file exists
if _, err := os.Stat(testLogFile); os.IsNotExist(err) {
b.Skip("Test log file not found:", testLogFile)
}
@@ -25,19 +21,16 @@ func BenchmarkOriginalLogParsing(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
_, err := GetLogLinesWithLimit("sshd", "", 100)
_, err := GetLogLinesWithLimit(context.Background(), "sshd", "", 100)
if err != nil {
b.Fatal(err)
}
}
}
// BenchmarkOptimizedLogParsing benchmarks the new optimized implementation
// BenchmarkOptimizedLogParsing benchmarks the simplified optimized entrypoint
func BenchmarkOptimizedLogParsing(b *testing.B) {
// Set up test environment with test data
testLogFile := filepath.Join("testdata", "fail2ban_full.log")
// Ensure test file exists
if _, err := os.Stat(testLogFile); os.IsNotExist(err) {
b.Skip("Test log file not found:", testLogFile)
}
@@ -56,325 +49,23 @@ func BenchmarkOptimizedLogParsing(b *testing.B) {
}
}
// BenchmarkGzipDetectionComparison compares gzip detection methods
func BenchmarkGzipDetectionComparison(b *testing.B) {
testFiles := []string{
filepath.Join("testdata", "fail2ban_full.log"), // Regular file
filepath.Join("testdata", "fail2ban_compressed.log.gz"), // Gzip file
}
processor := NewOptimizedLogProcessor()
for _, testFile := range testFiles {
if _, err := os.Stat(testFile); os.IsNotExist(err) {
continue // Skip if file doesn't exist
}
baseName := filepath.Base(testFile)
b.Run("original_"+baseName, func(b *testing.B) {
b.ResetTimer()
b.ReportAllocs()
for i := 0; i < b.N; i++ {
_, err := IsGzipFile(testFile)
if err != nil {
b.Fatal(err)
}
}
})
b.Run("optimized_"+baseName, func(b *testing.B) {
b.ResetTimer()
b.ReportAllocs()
for i := 0; i < b.N; i++ {
_ = processor.isGzipFileOptimized(testFile)
}
})
}
}
// BenchmarkFileNumberExtraction compares log number extraction methods
func BenchmarkFileNumberExtraction(b *testing.B) {
testFilenames := []string{
"fail2ban.log.1",
"fail2ban.log.2.gz",
"fail2ban.log.10",
"fail2ban.log.100.gz",
"fail2ban.log", // No number
}
processor := NewOptimizedLogProcessor()
b.Run("original", func(b *testing.B) {
b.ResetTimer()
b.ReportAllocs()
for i := 0; i < b.N; i++ {
for _, filename := range testFilenames {
_ = extractLogNumber(filename)
}
}
})
b.Run("optimized", func(b *testing.B) {
b.ResetTimer()
b.ReportAllocs()
for i := 0; i < b.N; i++ {
for _, filename := range testFilenames {
_ = processor.extractLogNumberOptimized(filename)
}
}
})
}
// BenchmarkLogFiltering compares log filtering performance
func BenchmarkLogFiltering(b *testing.B) {
// Sample log lines with various patterns
testLines := []string{
"2025-07-20 14:30:39,123 fail2ban.actions[1234]: NOTICE [sshd] Ban 192.168.1.100",
"2025-07-20 14:31:15,456 fail2ban.actions[1234]: NOTICE [apache] Ban 10.0.0.50",
"2025-07-20 14:32:01,789 fail2ban.filter[5678]: INFO [sshd] Found 192.168.1.100 - 2025-07-20 14:32:01",
"2025-07-20 14:33:45,012 fail2ban.actions[1234]: NOTICE [nginx] Ban 172.16.0.100",
"2025-07-20 14:34:22,345 fail2ban.filter[5678]: INFO [apache] Found 10.0.0.50 - 2025-07-20 14:34:22",
}
processor := NewOptimizedLogProcessor()
b.Run("original_jail_filter", func(b *testing.B) {
b.ResetTimer()
b.ReportAllocs()
for i := 0; i < b.N; i++ {
for _, line := range testLines {
// Simulate original filtering logic
_ = strings.Contains(line, "[sshd]")
}
}
})
b.Run("optimized_jail_filter", func(b *testing.B) {
b.ResetTimer()
b.ReportAllocs()
for i := 0; i < b.N; i++ {
for _, line := range testLines {
_ = processor.matchesFiltersOptimized(line, "sshd", "", true, false)
}
}
})
b.Run("original_ip_filter", func(b *testing.B) {
b.ResetTimer()
b.ReportAllocs()
for i := 0; i < b.N; i++ {
for _, line := range testLines {
// Simulate original IP filtering logic
_ = strings.Contains(line, "192.168.1.100")
}
}
})
b.Run("optimized_ip_filter", func(b *testing.B) {
b.ResetTimer()
b.ReportAllocs()
for i := 0; i < b.N; i++ {
for _, line := range testLines {
_ = processor.matchesFiltersOptimized(line, "", "192.168.1.100", false, true)
}
}
})
}
// BenchmarkCachePerformance tests the effectiveness of caching
func BenchmarkCachePerformance(b *testing.B) {
processor := NewOptimizedLogProcessor()
testFile := filepath.Join("testdata", "fail2ban_full.log")
if _, err := os.Stat(testFile); os.IsNotExist(err) {
b.Skip("Test file not found:", testFile)
}
b.Run("first_access_cache_miss", func(b *testing.B) {
b.ResetTimer()
b.ReportAllocs()
for i := 0; i < b.N; i++ {
processor.ClearCaches() // Clear cache to force miss
_ = processor.isGzipFileOptimized(testFile)
}
})
b.Run("repeated_access_cache_hit", func(b *testing.B) {
// Prime the cache
_ = processor.isGzipFileOptimized(testFile)
b.ResetTimer()
b.ReportAllocs()
for i := 0; i < b.N; i++ {
_ = processor.isGzipFileOptimized(testFile)
}
})
}
// BenchmarkStringPooling tests the effectiveness of string pooling
func BenchmarkStringPooling(b *testing.B) {
processor := NewOptimizedLogProcessor()
b.Run("with_pooling", func(b *testing.B) {
b.ResetTimer()
b.ReportAllocs()
for i := 0; i < b.N; i++ {
// Simulate getting and returning pooled slice
linesPtr := processor.stringPool.Get().(*[]string)
lines := (*linesPtr)[:0]
// Simulate adding lines
for j := 0; j < 100; j++ {
lines = append(lines, "test line")
}
// Return to pool
*linesPtr = lines[:0]
processor.stringPool.Put(linesPtr)
}
})
b.Run("without_pooling", func(b *testing.B) {
b.ResetTimer()
b.ReportAllocs()
for i := 0; i < b.N; i++ {
// Simulate creating new slice each time
lines := make([]string, 0, 1000)
// Simulate adding lines
for j := 0; j < 100; j++ {
lines = append(lines, "test line")
}
// Let it be garbage collected
_ = lines
}
})
}
// BenchmarkLargeLogDataset tests performance with larger datasets
func BenchmarkLargeLogDataset(b *testing.B) {
testLogFile := filepath.Join("testdata", "fail2ban_full.log")
if _, err := os.Stat(testLogFile); os.IsNotExist(err) {
b.Skip("Test log file not found:", testLogFile)
}
cleanup := setupBenchmarkLogEnvironment(b, testLogFile)
defer cleanup()
// Test with different line limits
limits := []int{100, 500, 1000, 5000}
for _, limit := range limits {
b.Run(fmt.Sprintf("original_lines_%d", limit), func(b *testing.B) {
b.ResetTimer()
b.ReportAllocs()
for i := 0; i < b.N; i++ {
_, err := GetLogLinesWithLimit("", "", limit)
if err != nil {
b.Fatal(err)
}
}
})
b.Run(fmt.Sprintf("optimized_lines_%d", limit), func(b *testing.B) {
b.ResetTimer()
b.ReportAllocs()
for i := 0; i < b.N; i++ {
_, err := GetLogLinesUltraOptimized("", "", limit)
if err != nil {
b.Fatal(err)
}
}
})
}
}
// BenchmarkMemoryPoolEfficiency tests memory pool efficiency
func BenchmarkMemoryPoolEfficiency(b *testing.B) {
processor := NewOptimizedLogProcessor()
// Test scanner buffer pooling
b.Run("scanner_buffer_pooling", func(b *testing.B) {
b.ResetTimer()
b.ReportAllocs()
for i := 0; i < b.N; i++ {
bufPtr := processor.scannerPool.Get().(*[]byte)
buf := (*bufPtr)[:cap(*bufPtr)]
// Simulate using buffer
for j := 0; j < 1000; j++ {
if j < len(buf) {
buf[j] = byte(j % 256)
}
}
*bufPtr = (*bufPtr)[:0]
processor.scannerPool.Put(bufPtr)
}
})
// Test line buffer pooling
b.Run("line_buffer_pooling", func(b *testing.B) {
b.ResetTimer()
b.ReportAllocs()
for i := 0; i < b.N; i++ {
bufPtr := processor.linePool.Get().(*[]byte)
buf := (*bufPtr)[:0]
// Simulate building a line
testLine := "test log line with some content"
buf = append(buf, testLine...)
*bufPtr = buf[:0]
processor.linePool.Put(bufPtr)
}
})
}
// Helper function to set up test environment (reuse from existing tests)
func setupBenchmarkLogEnvironment(tb testing.TB, testLogFile string) func() {
tb.Helper()
// Create temporary directory
tempDir := tb.TempDir()
// Copy test file to temp directory as fail2ban.log
mainLog := filepath.Join(tempDir, "fail2ban.log")
// Read and copy file
// #nosec G304 - testLogFile is a controlled test data file path
data, err := os.ReadFile(testLogFile)
func setupBenchmarkLogEnvironment(b *testing.B, source string) func() {
b.Helper()
data, err := os.ReadFile(source) // #nosec G304 // Reading a test file
if err != nil {
tb.Fatalf("Failed to read test file: %v", err)
b.Fatalf("failed to read test log file: %v", err)
}
if err := os.WriteFile(mainLog, data, 0600); err != nil {
tb.Fatalf("Failed to create test log: %v", err)
tempDir := b.TempDir()
dest := filepath.Join(tempDir, "fail2ban.log")
if err := os.WriteFile(dest, data, 0o600); err != nil {
b.Fatalf("failed to create benchmark log file: %v", err)
}
// Set log directory
origLogDir := GetLogDir()
origDir := GetLogDir()
SetLogDir(tempDir)
return func() {
SetLogDir(origLogDir)
SetLogDir(origDir)
}
}

136
fail2ban/fail2ban_log_performance_race_test.go
View File

@@ -1,136 +0,0 @@
package fail2ban
import (
"sync"
"testing"
)
func TestOptimizedLogProcessor_ConcurrentCacheAccess(t *testing.T) {
processor := NewOptimizedLogProcessor()
// Number of goroutines and operations per goroutine
numGoroutines := 100
opsPerGoroutine := 100
var wg sync.WaitGroup
// Start multiple goroutines that increment cache statistics
for i := 0; i < numGoroutines; i++ {
wg.Add(1)
go func() {
defer wg.Done()
for j := 0; j < opsPerGoroutine; j++ {
// Simulate cache hits and misses
processor.cacheHits.Add(1)
processor.cacheMisses.Add(1)
// Also read the stats
hits, misses := processor.GetCacheStats()
// Ensure values are monotonically increasing
if hits < 0 || misses < 0 {
t.Errorf("Cache stats should not be negative: hits=%d, misses=%d", hits, misses)
}
}
}()
}
wg.Wait()
// Verify final counts
finalHits, finalMisses := processor.GetCacheStats()
expectedCount := int64(numGoroutines * opsPerGoroutine)
if finalHits != expectedCount {
t.Errorf("Expected %d cache hits, got %d", expectedCount, finalHits)
}
if finalMisses != expectedCount {
t.Errorf("Expected %d cache misses, got %d", expectedCount, finalMisses)
}
}
func TestOptimizedLogProcessor_ConcurrentCacheClear(t *testing.T) {
processor := NewOptimizedLogProcessor()
// Number of goroutines
numGoroutines := 50
var wg sync.WaitGroup
// Start goroutines that increment stats and clear caches concurrently
for i := 0; i < numGoroutines; i++ {
wg.Add(1)
go func(id int) {
defer wg.Done()
// Half increment, half clear
if id%2 == 0 {
// Incrementer goroutines
for j := 0; j < 100; j++ {
processor.cacheHits.Add(1)
processor.cacheMisses.Add(1)
}
} else {
// Clearer goroutines
for j := 0; j < 10; j++ {
processor.ClearCaches()
}
}
}(i)
}
wg.Wait()
// Test should complete without races - exact final values don't matter
// since clears can happen at any time
hits, misses := processor.GetCacheStats()
// Values should be non-negative
if hits < 0 || misses < 0 {
t.Errorf("Cache stats should not be negative after concurrent operations: hits=%d, misses=%d", hits, misses)
}
}
func TestOptimizedLogProcessor_CacheStatsConsistency(t *testing.T) {
processor := NewOptimizedLogProcessor()
// Test initial state
hits, misses := processor.GetCacheStats()
if hits != 0 || misses != 0 {
t.Errorf("Initial cache stats should be zero: hits=%d, misses=%d", hits, misses)
}
// Test increment operations
processor.cacheHits.Add(5)
processor.cacheMisses.Add(3)
hits, misses = processor.GetCacheStats()
if hits != 5 || misses != 3 {
t.Errorf("Cache stats after increment: expected hits=5, misses=3; got hits=%d, misses=%d", hits, misses)
}
// Test clear operation
processor.ClearCaches()
hits, misses = processor.GetCacheStats()
if hits != 0 || misses != 0 {
t.Errorf("Cache stats after clear should be zero: hits=%d, misses=%d", hits, misses)
}
}
func BenchmarkOptimizedLogProcessor_ConcurrentCacheStats(b *testing.B) {
processor := NewOptimizedLogProcessor()
b.RunParallel(func(pb *testing.PB) {
for pb.Next() {
// Simulate cache operations
processor.cacheHits.Add(1)
processor.cacheMisses.Add(1)
// Read stats
processor.GetCacheStats()
}
})
}

1
fail2ban/fail2ban_log_security_test.go
View File

@@ -33,6 +33,7 @@ func TestReadLogFileSecurityValidation(t *testing.T) {
"invalid path",
"not in expected system location",
"outside allowed directories",
"null byte",
},
) {
t.Errorf("Error should be security-related, got: %s", errorMsg)

24
fail2ban/fail2ban_logs_integration_test.go
View File

@@ -28,7 +28,7 @@ func TestIntegrationFullLogProcessing(t *testing.T) {
// testProcessFullLog tests processing of the entire log file
func testProcessFullLog(t *testing.T) {
start := time.Now()
lines, err := GetLogLines("", "")
lines, err := GetLogLines(context.Background(), "", "")
duration := time.Since(start)
if err != nil {
@@ -50,7 +50,7 @@ func testProcessFullLog(t *testing.T) {
// testExtractBanEvents tests extraction of ban/unban events
func testExtractBanEvents(t *testing.T) {
lines, err := GetLogLines("sshd", "")
lines, err := GetLogLines(context.Background(), "sshd", "")
if err != nil {
t.Fatalf("Failed to get log lines: %v", err)
}
@@ -74,7 +74,7 @@ func testExtractBanEvents(t *testing.T) {
// testTrackPersistentAttacker tests tracking a specific attacker across the log
func testTrackPersistentAttacker(t *testing.T) {
// Track 192.168.1.100 (most frequent attacker)
lines, err := GetLogLines("", "192.168.1.100")
lines, err := GetLogLines(context.Background(), "", "192.168.1.100")
if err != nil {
t.Fatalf("Failed to filter by IP: %v", err)
}
@@ -157,7 +157,7 @@ func TestIntegrationConcurrentLogReading(t *testing.T) {
ip = "10.0.0.50"
}
lines, err := GetLogLines(jail, ip)
lines, err := GetLogLines(context.Background(), jail, ip)
if err != nil {
errors <- err
return
@@ -182,7 +182,10 @@ func TestIntegrationConcurrentLogReading(t *testing.T) {
func TestIntegrationBanRecordParsing(t *testing.T) {
// Test parsing ban records with real patterns
parser := NewBanRecordParser()
parser, err := NewBanRecordParser()
if err != nil {
t.Fatal(err)
}
// Use dynamic dates relative to current time
now := time.Now()
@@ -304,7 +307,7 @@ func TestIntegrationParallelLogProcessing(t *testing.T) {
start := time.Now()
results, err := pool.Process(ctx, jails, func(_ context.Context, jail string) ([]string, error) {
return GetLogLines(jail, "")
return GetLogLines(context.Background(), jail, "")
})
duration := time.Since(start)
@@ -349,7 +352,7 @@ func TestIntegrationMemoryUsage(t *testing.T) {
// Process log multiple times to check for leaks
for i := 0; i < 10; i++ {
lines, err := GetLogLines("", "")
lines, err := GetLogLines(context.Background(), "", "")
if err != nil {
t.Fatalf("Iteration %d failed: %v", i, err)
}
@@ -425,7 +428,7 @@ func BenchmarkLogParsing(b *testing.B) {
b.ResetTimer()
for i := 0; i < b.N; i++ {
_, err := GetLogLines("sshd", "")
_, err := GetLogLines(context.Background(), "sshd", "")
if err != nil {
b.Fatalf("Benchmark failed: %v", err)
}
@@ -433,7 +436,10 @@ func BenchmarkLogParsing(b *testing.B) {
}
func BenchmarkBanRecordParsing(b *testing.B) {
parser := NewBanRecordParser()
parser, err := NewBanRecordParser()
if err != nil {
b.Fatal(err)
}
// Use dynamic dates for benchmark
now := time.Now()

16
fail2ban/fail2ban_logs_parsing_test.go
View File

@@ -1,6 +1,7 @@
package fail2ban
import (
"context"
"errors"
"os"
"path/filepath"
@@ -8,6 +9,8 @@ import (
"strings"
"testing"
"time"
"github.com/ivuorinen/f2b/shared"
)
// parseTimestamp extracts and parses timestamp from log line
@@ -243,7 +246,7 @@ func TestGetLogLinesWithRealTestData(t *testing.T) {
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
lines, err := GetLogLines(tt.jail, tt.ip)
lines, err := GetLogLines(context.Background(), tt.jail, tt.ip)
if err != nil {
t.Fatalf("GetLogLines failed: %v", err)
}
@@ -270,7 +273,10 @@ func TestGetLogLinesWithRealTestData(t *testing.T) {
func TestParseBanRecordsFromRealLogs(t *testing.T) {
// Test with real ban/unban patterns from production
parser := NewBanRecordParser()
parser, err := NewBanRecordParser()
if err != nil {
t.Fatal(err)
}
tests := []struct {
name string
@@ -342,7 +348,7 @@ func TestLogFileRotationPatterns(t *testing.T) {
for _, file := range testFiles {
path := filepath.Join(tempDir, file)
if strings.HasSuffix(file, ".gz") {
if strings.HasSuffix(file, shared.GzipExtension) {
// Create compressed file
content := []byte("test log content")
createTestGzipFile(t, path, content)
@@ -380,7 +386,7 @@ func TestMalformedLogHandling(t *testing.T) {
defer cleanup()
// Should handle malformed entries gracefully
lines, err := GetLogLines("", "")
lines, err := GetLogLines(context.Background(), "", "")
if err != nil {
t.Fatalf("GetLogLines should handle malformed entries: %v", err)
}
@@ -416,7 +422,7 @@ func TestMultiJailLogParsing(t *testing.T) {
for _, jail := range jails {
t.Run("jail_"+jail, func(t *testing.T) {
lines, err := GetLogLines(jail, "")
lines, err := GetLogLines(context.Background(), jail, "")
if err != nil {
t.Fatalf("GetLogLines failed for jail %s: %v", jail, err)
}

14
fail2ban/fail2ban_path_security_test.go
View File

@@ -36,7 +36,7 @@ func TestPathTraversalDetection(t *testing.T) {
for _, maliciousPath := range maliciousPaths {
t.Run("malicious_path", func(t *testing.T) {
_, err := validatePathWithSecurity(maliciousPath, config)
_, err := ValidatePathWithSecurity(maliciousPath, config)
if err == nil {
t.Errorf("expected error for malicious path %q, but validation passed", maliciousPath)
}
@@ -71,7 +71,7 @@ func TestValidPaths(t *testing.T) {
for _, validPath := range validPaths {
t.Run("valid_path", func(t *testing.T) {
result, err := validatePathWithSecurity(validPath, config)
result, err := ValidatePathWithSecurity(validPath, config)
if err != nil {
t.Errorf("expected valid path %q to pass validation, got error: %v", validPath, err)
}
@@ -112,7 +112,7 @@ func TestSymlinkHandling(t *testing.T) {
ResolveSymlinks: true,
}
_, err := validatePathWithSecurity(symlinkPath, configNoSymlinks)
_, err := ValidatePathWithSecurity(symlinkPath, configNoSymlinks)
if err == nil {
t.Error("expected error for symlink when symlinks are disabled")
}
@@ -125,7 +125,7 @@ func TestSymlinkHandling(t *testing.T) {
ResolveSymlinks: true,
}
_, err = validatePathWithSecurity(symlinkPath, configWithSymlinks)
_, err = ValidatePathWithSecurity(symlinkPath, configWithSymlinks)
if err == nil {
t.Error("expected error for symlink pointing outside allowed directory")
}
@@ -227,7 +227,7 @@ func TestPathLengthLimits(t *testing.T) {
ResolveSymlinks: true,
}
_, err := validatePathWithSecurity(normalPath, config)
_, err := ValidatePathWithSecurity(normalPath, config)
if err != nil {
t.Errorf("normal length path should pass: %v", err)
}
@@ -236,7 +236,7 @@ func TestPathLengthLimits(t *testing.T) {
longName := strings.Repeat("a", 5000)
longPath := filepath.Join(tempDir, longName)
_, err = validatePathWithSecurity(longPath, config)
_, err = ValidatePathWithSecurity(longPath, config)
if err == nil {
t.Error("extremely long path should fail validation")
}
@@ -342,7 +342,7 @@ func BenchmarkPathValidation(b *testing.B) {
b.ResetTimer()
for i := 0; i < b.N; i++ {
_, err := validatePathWithSecurity(testPath, config)
_, err := ValidatePathWithSecurity(testPath, config)
if err != nil {
b.Fatalf("unexpected error: %v", err)
}

58
fail2ban/fail2ban_time_parser_test.go
View File

@@ -3,10 +3,18 @@ package fail2ban
import (
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/ivuorinen/f2b/shared"
)
func TestTimeParsingCache(t *testing.T) {
cache := NewTimeParsingCache("2006-01-02 15:04:05")
cache, err := NewTimeParsingCache("2006-01-02 15:04:05")
if err != nil {
t.Fatal(err)
}
// Test basic parsing
testTime := "2023-12-01 14:30:45"
@@ -33,7 +41,10 @@ func TestTimeParsingCache(t *testing.T) {
}
func TestBuildTimeString(t *testing.T) {
cache := NewTimeParsingCache("2006-01-02 15:04:05")
cache, err := NewTimeParsingCache("2006-01-02 15:04:05")
if err != nil {
t.Fatal(err)
}
result := cache.BuildTimeString("2023-12-01", "14:30:45")
expected := "2023-12-01 14:30:45"
@@ -66,7 +77,11 @@ func TestBuildBanTimeString(t *testing.T) {
}
func BenchmarkTimeParsingWithCache(b *testing.B) {
cache := NewTimeParsingCache("2006-01-02 15:04:05")
cache, err := NewTimeParsingCache("2006-01-02 15:04:05")
if err != nil {
b.Fatal(err)
}
testTime := "2023-12-01 14:30:45"
b.ResetTimer()
@@ -86,7 +101,10 @@ func BenchmarkTimeParsingWithoutCache(b *testing.B) {
}
func BenchmarkBuildTimeString(b *testing.B) {
cache := NewTimeParsingCache("2006-01-02 15:04:05")
cache, err := NewTimeParsingCache("2006-01-02 15:04:05")
if err != nil {
b.Fatal(err)
}
b.ResetTimer()
for i := 0; i < b.N; i++ {
@@ -100,3 +118,35 @@ func BenchmarkBuildTimeStringNaive(b *testing.B) {
_ = "2023-12-01" + " " + "14:30:45"
}
}
// TestTimeParsingCache_BoundedEviction verifies that the cache doesn't grow unbounded
func TestTimeParsingCache_BoundedEviction(t *testing.T) {
cache, err := NewTimeParsingCache("2006-01-02 15:04:05")
if err != nil {
t.Fatal(err)
}
// Add significantly more than max to ensure eviction triggers
entriesToAdd := shared.CacheMaxSize + 1000
// Create base time for monotonic timestamp generation
baseTime := time.Date(2024, 1, 1, 0, 0, 0, 0, time.UTC)
for i := 0; i < entriesToAdd; i++ {
// Generate unique time strings using monotonic increment
uniqueTime := baseTime.Add(time.Duration(i) * time.Second)
timeStr := uniqueTime.Format("2006-01-02 15:04:05")
_, err := cache.ParseTime(timeStr)
require.NoError(t, err)
}
// Verify cache was evicted and didn't grow unbounded
size := cache.parseCache.Size()
assert.LessOrEqual(t, size, shared.CacheMaxSize,
"Cache must not exceed max size after eviction")
assert.Greater(t, size, 0,
"Cache should still contain entries after eviction")
t.Logf("Cache size after adding %d entries: %d (max: %d, evicted: %d)",
entriesToAdd, size, shared.CacheMaxSize, entriesToAdd-size)
}

40
fail2ban/fail2ban_utils_test.go
View File

@@ -4,6 +4,7 @@ package fail2ban_test
import (
"compress/gzip"
"context"
"fmt"
"os"
"path/filepath"
@@ -11,6 +12,8 @@ import (
"testing"
"time"
"github.com/ivuorinen/f2b/shared"
"github.com/ivuorinen/f2b/fail2ban"
)
@@ -32,7 +35,7 @@ func TestSetLogDir(t *testing.T) {
err := os.WriteFile(filepath.Join(tempDir, "fail2ban.log"), []byte(logContent), 0600)
fail2ban.AssertError(t, err, false, "create test log file")
lines, err := fail2ban.GetLogLines("", "")
lines, err := fail2ban.GetLogLines(context.Background(), "", "")
fail2ban.AssertError(t, err, false, "GetLogLines")
if len(lines) != 1 || lines[0] != logContent {
@@ -82,13 +85,18 @@ func TestOSRunnerWithoutSudo(t *testing.T) {
// TestOSRunnerWithSudo tests the OS runner with sudo
func TestOSRunnerWithSudo(t *testing.T) {
runner := &fail2ban.OSRunner{}
// Test with a command that would use sudo
// Note: This might fail in CI/test environments without sudo
_, err := runner.CombinedOutput("sudo", "echo", "hello")
if err != nil {
t.Logf("sudo command failed as expected in test environment: %v", err)
// Do not parallelize: this test mutates global runner
orig := fail2ban.GetRunner()
t.Cleanup(func() { fail2ban.SetRunner(orig) })
mock := &fail2ban.MockRunner{
Responses: map[string][]byte{"sudo echo hello": []byte("hello\n")},
Errors: map[string]error{},
}
fail2ban.SetRunner(mock)
out, err := fail2ban.RunnerCombinedOutput("sudo", "echo", "hello")
fail2ban.AssertError(t, err, false, "RunnerCombinedOutput with sudo (mocked)")
if strings.TrimSpace(string(out)) != "hello" {
t.Fatalf("expected %q, got %q", "hello", strings.TrimSpace(string(out)))
}
}
@@ -194,7 +202,7 @@ func TestLogFileReading(t *testing.T) {
}
// Test reading
lines, err := fail2ban.GetLogLines("", "")
lines, err := fail2ban.GetLogLines(context.Background(), "", "")
fail2ban.AssertError(t, err, false, tt.name)
validateLogLines(t, lines, tt.expected, tt.name)
@@ -222,7 +230,7 @@ func TestLogFileOrdering(t *testing.T) {
}
}
lines, err := fail2ban.GetLogLines("", "")
lines, err := fail2ban.GetLogLines(context.Background(), "", "")
fail2ban.AssertError(t, err, false, "GetLogLines ordering test")
// Should be in chronological order: oldest rotated first, then current
@@ -316,7 +324,7 @@ func TestLogFiltering(t *testing.T) {
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
lines, err := fail2ban.GetLogLines(tt.jailFilter, tt.ipFilter)
lines, err := fail2ban.GetLogLines(context.Background(), tt.jailFilter, tt.ipFilter)
fail2ban.AssertError(t, err, false, tt.name)
if len(lines) != tt.expectedCount {
@@ -348,7 +356,7 @@ func TestBanRecordFormatting(t *testing.T) {
fail2ban.SetRunner(mock)
client, err := fail2ban.NewClient(fail2ban.DefaultLogDir, fail2ban.DefaultFilterDir)
client, err := fail2ban.NewClient(shared.DefaultLogDir, shared.DefaultFilterDir)
fail2ban.AssertError(t, err, false, "create client")
records, err := client.GetBanRecords([]string{"sshd"})
@@ -440,7 +448,7 @@ func TestVersionComparisonEdgeCases(t *testing.T) {
}
fail2ban.SetRunner(mock)
_, err := fail2ban.NewClient(fail2ban.DefaultLogDir, fail2ban.DefaultFilterDir)
_, err := fail2ban.NewClient(shared.DefaultLogDir, shared.DefaultFilterDir)
fail2ban.AssertError(t, err, tt.expectError, tt.name)
})
@@ -503,7 +511,7 @@ func TestClientInitializationEdgeCases(t *testing.T) {
tt.setupMock(mock)
fail2ban.SetRunner(mock)
_, err := fail2ban.NewClient(fail2ban.DefaultLogDir, fail2ban.DefaultFilterDir)
_, err := fail2ban.NewClient(shared.DefaultLogDir, shared.DefaultFilterDir)
fail2ban.AssertError(t, err, tt.expectError, tt.name)
if tt.expectError && tt.errorMsg != "" {
@@ -527,7 +535,7 @@ func TestConcurrentAccess(t *testing.T) {
mock.SetResponse("fail2ban-client banned 192.168.1.100", []byte(`["sshd"]`))
fail2ban.SetRunner(mock)
client, err := fail2ban.NewClient(fail2ban.DefaultLogDir, fail2ban.DefaultFilterDir)
client, err := fail2ban.NewClient(shared.DefaultLogDir, shared.DefaultFilterDir)
fail2ban.AssertError(t, err, false, "create client for concurrency test")
// Run concurrent operations
@@ -579,7 +587,7 @@ func TestMemoryUsage(t *testing.T) {
// Create and destroy many clients
for i := 0; i < 1000; i++ {
client, err := fail2ban.NewClient(fail2ban.DefaultLogDir, fail2ban.DefaultFilterDir)
client, err := fail2ban.NewClient(shared.DefaultLogDir, shared.DefaultFilterDir)
fail2ban.AssertError(t, err, false, "create client in memory test")
// Use the client

24
fail2ban/gzip_detection.go
View File

@@ -7,6 +7,8 @@ import (
"io"
"os"
"strings"
"github.com/ivuorinen/f2b/shared"
)
// GzipDetector provides utilities for detecting and handling gzip-compressed files
@@ -21,7 +23,7 @@ func NewGzipDetector() *GzipDetector {
// then falling back to magic byte detection for better performance
func (gd *GzipDetector) IsGzipFile(path string) (bool, error) {
// Fast path: check file extension first
if strings.HasSuffix(strings.ToLower(path), ".gz") {
if strings.HasSuffix(strings.ToLower(path), shared.GzipExtension) {
return true, nil
}
@@ -39,7 +41,7 @@ func (gd *GzipDetector) hasGzipMagicBytes(path string) (bool, error) {
defer func() {
if closeErr := f.Close(); closeErr != nil {
getLogger().WithError(closeErr).
WithField("path", path).
WithField(shared.LogFieldFile, path).
Warn("Failed to close file in gzip magic byte check")
}
}()
@@ -51,7 +53,11 @@ func (gd *GzipDetector) hasGzipMagicBytes(path string) (bool, error) {
}
// Check if we have gzip magic bytes (0x1f, 0x8b)
return n >= 2 && magic[0] == 0x1f && magic[1] == 0x8b, nil
if n < 2 {
return false, nil
}
// #nosec G602 - Length check above guarantees slice has at least 2 elements
return magic[0] == 0x1f && magic[1] == 0x8b, nil
}
// OpenGzipAwareReader opens a file and returns appropriate reader (gzip or regular)
@@ -65,7 +71,9 @@ func (gd *GzipDetector) OpenGzipAwareReader(path string) (io.ReadCloser, error)
isGzip, err := gd.IsGzipFile(path)
if err != nil {
if closeErr := f.Close(); closeErr != nil {
getLogger().WithError(closeErr).WithField("file", path).Warn("Failed to close file during error handling")
getLogger().WithError(closeErr).
WithField(shared.LogFieldFile, path).
Warn("Failed to close file during error handling")
}
return nil, err
}
@@ -76,7 +84,7 @@ func (gd *GzipDetector) OpenGzipAwareReader(path string) (io.ReadCloser, error)
if err != nil {
if closeErr := f.Close(); closeErr != nil {
getLogger().WithError(closeErr).
WithField("file", path).
WithField(shared.LogFieldFile, path).
Warn("Failed to close file during seek error handling")
}
return nil, err
@@ -86,7 +94,7 @@ func (gd *GzipDetector) OpenGzipAwareReader(path string) (io.ReadCloser, error)
if err != nil {
if closeErr := f.Close(); closeErr != nil {
getLogger().WithError(closeErr).
WithField("file", path).
WithField(shared.LogFieldFile, path).
Warn("Failed to close file during gzip reader error handling")
}
return nil, err
@@ -121,7 +129,9 @@ func (gd *GzipDetector) CreateGzipAwareScannerWithBuffer(path string, maxLineSiz
cleanup := func() {
if err := reader.Close(); err != nil {
getLogger().WithError(err).WithField("file", path).Warn("Failed to close reader during cleanup")
getLogger().WithError(err).
WithField(shared.LogFieldFile, path).
Warn("Failed to close reader during cleanup")
}
}

879
fail2ban/helpers.go
View File

File diff suppressed because it is too large Load Diff

2
fail2ban/helpers_additional_test.go
View File

@@ -136,7 +136,7 @@ func TestValidationCacheSize(t *testing.T) {
}
// Add something to cache
err := CachedValidateIP("192.168.1.1")
err := CachedValidateIP(context.Background(), "192.168.1.1")
if err != nil {
t.Fatalf("CachedValidateIP failed: %v", err)
}

216
fail2ban/helpers_validation_test.go Normal file
View File

@@ -0,0 +1,216 @@
package fail2ban
import (
"context"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// TestValidateFilterName tests the ValidateFilterName function
func TestValidateFilterName(t *testing.T) {
tests := []struct {
name string
filter string
expectError bool
errorMsg string
}{
{
name: "valid filter name",
filter: "sshd",
expectError: false,
},
{
name: "valid filter name with dash",
filter: "sshd-aggressive",
expectError: false,
},
{
name: "empty filter name",
filter: "",
expectError: true,
errorMsg: "filter name cannot be empty",
},
{
name: "filter name with spaces gets trimmed",
filter: " sshd ",
expectError: false,
},
{
name: "filter name with path traversal",
filter: "../../../etc/passwd",
expectError: true,
errorMsg: "filter name contains path traversal",
},
{
name: "filter name with dot dot - caught by character validation",
filter: "filter..conf",
expectError: true,
errorMsg: "filter name contains invalid characters",
},
{
name: "absolute path filter name - caught by path traversal first",
filter: "/etc/fail2ban/filter.d/sshd.conf",
expectError: true,
errorMsg: "filter name contains path traversal",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := ValidateFilterName(tt.filter)
if tt.expectError {
assert.Error(t, err)
if tt.errorMsg != "" {
assert.Contains(t, err.Error(), tt.errorMsg)
}
} else {
assert.NoError(t, err)
}
})
}
}
// TestGetLogLinesWrapper tests the GetLogLines wrapper function
func TestGetLogLinesWrapper(t *testing.T) {
// Save and restore original runner
originalRunner := GetRunner()
defer SetRunner(originalRunner)
mockRunner := NewMockRunner()
mockRunner.SetResponse("fail2ban-client -V", []byte("Fail2Ban v0.11.0"))
mockRunner.SetResponse("sudo fail2ban-client -V", []byte("Fail2Ban v0.11.0"))
mockRunner.SetResponse("fail2ban-client ping", []byte("Server replied: pong"))
mockRunner.SetResponse("sudo fail2ban-client ping", []byte("Server replied: pong"))
mockRunner.SetResponse("fail2ban-client status", []byte("Status\n|- Number of jail: 1\n`- Jail list: sshd"))
mockRunner.SetResponse("sudo fail2ban-client status", []byte("Status\n|- Number of jail: 1\n`- Jail list: sshd"))
SetRunner(mockRunner)
// Create temporary log directory
tmpDir := t.TempDir()
oldLogDir := GetLogDir()
SetLogDir(tmpDir)
defer SetLogDir(oldLogDir)
client, err := NewClient("/var/log/fail2ban", "/etc/fail2ban/filter.d")
require.NoError(t, err)
// Call GetLogLines (wrapper for GetLogLinesWithLimit)
lines, err := client.GetLogLines("sshd", "192.168.1.1")
// May return error if no log files exist, which is ok
_ = err
_ = lines
}
// TestBanIPWithContext tests the BanIPWithContext function
func TestBanIPWithContext(t *testing.T) {
// Save and restore original runner
originalRunner := GetRunner()
defer SetRunner(originalRunner)
tests := []struct {
name string
setupMock func(*MockRunner)
ip string
jail string
expectError bool
}{
{
name: "successful ban",
setupMock: func(m *MockRunner) {
m.SetResponse("fail2ban-client -V", []byte("Fail2Ban v0.11.0"))
m.SetResponse("sudo fail2ban-client -V", []byte("Fail2Ban v0.11.0"))
m.SetResponse("fail2ban-client ping", []byte("Server replied: pong"))
m.SetResponse("sudo fail2ban-client ping", []byte("Server replied: pong"))
m.SetResponse("fail2ban-client status", []byte("Status\n|- Number of jail: 1\n`- Jail list: sshd"))
m.SetResponse("sudo fail2ban-client status", []byte("Status\n|- Number of jail: 1\n`- Jail list: sshd"))
m.SetResponse("fail2ban-client set sshd banip 192.168.1.1", []byte("1"))
m.SetResponse("sudo fail2ban-client set sshd banip 192.168.1.1", []byte("1"))
},
ip: "192.168.1.1",
jail: "sshd",
expectError: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
mockRunner := NewMockRunner()
tt.setupMock(mockRunner)
SetRunner(mockRunner)
client, err := NewClient("/var/log/fail2ban", "/etc/fail2ban/filter.d")
require.NoError(t, err)
ctx := context.Background()
count, err := client.BanIPWithContext(ctx, tt.ip, tt.jail)
if tt.expectError {
assert.Error(t, err)
} else {
assert.NoError(t, err)
assert.GreaterOrEqual(t, count, 0, "Count should be 0 (new ban) or 1 (already banned)")
}
})
}
}
// TestGetLogLinesWithLimitAndContext tests the GetLogLinesWithLimitAndContext function
func TestGetLogLinesWithLimitAndContext(t *testing.T) {
// Save and restore original runner
originalRunner := GetRunner()
defer SetRunner(originalRunner)
mockRunner := NewMockRunner()
mockRunner.SetResponse("fail2ban-client -V", []byte("Fail2Ban v0.11.0"))
mockRunner.SetResponse("sudo fail2ban-client -V", []byte("Fail2Ban v0.11.0"))
mockRunner.SetResponse("fail2ban-client ping", []byte("Server replied: pong"))
mockRunner.SetResponse("sudo fail2ban-client ping", []byte("Server replied: pong"))
mockRunner.SetResponse("fail2ban-client status", []byte("Status\n|- Number of jail: 1\n`- Jail list: sshd"))
mockRunner.SetResponse("sudo fail2ban-client status", []byte("Status\n|- Number of jail: 1\n`- Jail list: sshd"))
SetRunner(mockRunner)
// Create temporary log directory
tmpDir := t.TempDir()
oldLogDir := GetLogDir()
SetLogDir(tmpDir)
defer SetLogDir(oldLogDir)
client, err := NewClient("/var/log/fail2ban", "/etc/fail2ban/filter.d")
require.NoError(t, err)
ctx := context.Background()
tests := []struct {
name string
jail string
ip string
maxLines int
}{
{
name: "get log lines with limit",
jail: "sshd",
ip: "192.168.1.1",
maxLines: 10,
},
{
name: "zero max lines",
jail: "sshd",
ip: "192.168.1.1",
maxLines: 0,
},
}
for _, tt := range tests {
t.Run(tt.name, func(_ *testing.T) {
lines, err := client.GetLogLinesWithLimitAndContext(ctx, tt.jail, tt.ip, tt.maxLines)
// May return error if no log files exist, which is ok for this test
_ = err
_ = lines
})
}
}

75
fail2ban/interfaces.go Normal file
View File

@@ -0,0 +1,75 @@
// Package fail2ban defines core interfaces and contracts for fail2ban operations.
// This package provides the primary interfaces (Client, Runner, SudoChecker) that
// define the contract for interacting with fail2ban services and system operations.
package fail2ban
import (
"context"
)
// Client defines the interface for interacting with Fail2Ban.
// Implementations must provide all core operations for jail and ban management.
type Client interface {
// ListJails returns all available Fail2Ban jails.
ListJails() ([]string, error)
// StatusAll returns the status output for all jails.
StatusAll() (string, error)
// StatusJail returns the status output for a specific jail.
StatusJail(string) (string, error)
// BanIP bans the given IP in the specified jail. Returns 0 if banned, 1 if already banned.
BanIP(ip, jail string) (int, error)
// UnbanIP unbans the given IP in the specified jail. Returns 0 if unbanned, 1 if already unbanned.
UnbanIP(ip, jail string) (int, error)
// BannedIn returns the list of jails in which the IP is currently banned.
BannedIn(ip string) ([]string, error)
// GetBanRecords returns ban records for the specified jails.
GetBanRecords(jails []string) ([]BanRecord, error)
// GetLogLines returns log lines filtered by jail and/or IP.
GetLogLines(jail, ip string) ([]string, error)
// ListFilters returns the available Fail2Ban filters.
ListFilters() ([]string, error)
// TestFilter runs fail2ban-regex for the given filter.
TestFilter(filter string) (string, error)
// Context-aware versions for timeout and cancellation support
ListJailsWithContext(ctx context.Context) ([]string, error)
StatusAllWithContext(ctx context.Context) (string, error)
StatusJailWithContext(ctx context.Context, jail string) (string, error)
BanIPWithContext(ctx context.Context, ip, jail string) (int, error)
UnbanIPWithContext(ctx context.Context, ip, jail string) (int, error)
BannedInWithContext(ctx context.Context, ip string) ([]string, error)
GetBanRecordsWithContext(ctx context.Context, jails []string) ([]BanRecord, error)
GetLogLinesWithContext(ctx context.Context, jail, ip string) ([]string, error)
ListFiltersWithContext(ctx context.Context) ([]string, error)
TestFilterWithContext(ctx context.Context, filter string) (string, error)
}
// Runner defines the interface for executing system commands.
// Implementations provide different execution strategies (real, mock, etc.).
type Runner interface {
CombinedOutput(name string, args ...string) ([]byte, error)
CombinedOutputWithSudo(name string, args ...string) ([]byte, error)
// Context-aware versions for timeout and cancellation support
CombinedOutputWithContext(ctx context.Context, name string, args ...string) ([]byte, error)
CombinedOutputWithSudoContext(ctx context.Context, name string, args ...string) ([]byte, error)
}
// SudoChecker provides methods to check sudo privileges
type SudoChecker interface {
// IsRoot returns true if the current user is root (UID 0)
IsRoot() bool
// InSudoGroup returns true if the current user is in the sudo group
InSudoGroup() bool
// CanUseSudo returns true if the current user can use sudo
CanUseSudo() bool
// HasSudoPrivileges returns true if user has any form of sudo access
HasSudoPrivileges() bool
}
// MetricsRecorder defines interface for recording metrics
type MetricsRecorder interface {
// RecordValidationCacheHit records validation cache hits
RecordValidationCacheHit()
// RecordValidationCacheMiss records validation cache misses
RecordValidationCacheMiss()
}

497
fail2ban/log_performance_optimized.go
View File

@@ -1,497 +0,0 @@
package fail2ban
import (
"bufio"
"fmt"
"os"
"path/filepath"
"sort"
"strconv"
"strings"
"sync"
"sync/atomic"
)
// OptimizedLogProcessor provides high-performance log processing with caching and optimizations
type OptimizedLogProcessor struct {
// Caches for performance
gzipCache sync.Map // string -> bool (path -> isGzip)
pathCache sync.Map // string -> string (pattern -> cleanPath)
fileInfoCache sync.Map // string -> *CachedFileInfo
// Object pools for reducing allocations
stringPool sync.Pool
linePool sync.Pool
scannerPool sync.Pool
// Statistics (thread-safe atomic counters)
cacheHits atomic.Int64
cacheMisses atomic.Int64
}
// CachedFileInfo holds cached information about a log file
type CachedFileInfo struct {
Path string
IsGzip bool
Size int64
ModTime int64
LogNumber int // For rotated logs: -1 for current, >=0 for rotated
IsValid bool
}
// OptimizedRotatedLog represents a rotated log file with cached info
type OptimizedRotatedLog struct {
Num int
Path string
Info *CachedFileInfo
}
// NewOptimizedLogProcessor creates a new high-performance log processor
func NewOptimizedLogProcessor() *OptimizedLogProcessor {
processor := &OptimizedLogProcessor{}
// String slice pool for lines
processor.stringPool = sync.Pool{
New: func() interface{} {
s := make([]string, 0, 1000) // Pre-allocate for typical log sizes
return &s
},
}
// Line buffer pool for individual lines
processor.linePool = sync.Pool{
New: func() interface{} {
b := make([]byte, 0, 512) // Pre-allocate for typical line lengths
return &b
},
}
// Scanner buffer pool
processor.scannerPool = sync.Pool{
New: func() interface{} {
b := make([]byte, 0, 64*1024) // 64KB scanner buffer
return &b
},
}
return processor
}
// GetLogLinesOptimized provides optimized log line retrieval with caching
func (olp *OptimizedLogProcessor) GetLogLinesOptimized(jailFilter, ipFilter string, maxLines int) ([]string, error) {
// Fast path for log directory pattern caching
pattern := filepath.Join(GetLogDir(), "fail2ban.log*")
files, err := olp.getCachedGlobResults(pattern)
if err != nil {
return nil, fmt.Errorf("error listing log files: %w", err)
}
if len(files) == 0 {
return []string{}, nil
}
// Optimized file parsing and sorting
currentLog, rotated := olp.parseLogFilesOptimized(files)
// Get pooled string slice
linesPtr := olp.stringPool.Get().(*[]string)
lines := (*linesPtr)[:0] // Reset slice but keep capacity
defer func() {
*linesPtr = lines[:0]
olp.stringPool.Put(linesPtr)
}()
config := LogReadConfig{
MaxLines: maxLines,
MaxFileSize: 100 * 1024 * 1024, // 100MB file size limit
JailFilter: jailFilter,
IPFilter: ipFilter,
ReverseOrder: false,
}
totalLines := 0
// Process rotated logs first (oldest to newest)
for _, rotatedLog := range rotated {
if config.MaxLines > 0 && totalLines >= config.MaxLines {
break
}
remainingLines := config.MaxLines - totalLines
if remainingLines <= 0 {
break
}
fileConfig := config
fileConfig.MaxLines = remainingLines
fileLines, err := olp.streamLogFileOptimized(rotatedLog.Path, fileConfig)
if err != nil {
getLogger().WithError(err).WithField("file", rotatedLog.Path).Error("Failed to read log file")
continue
}
lines = append(lines, fileLines...)
totalLines += len(fileLines)
}
// Process current log last
if currentLog != "" && (config.MaxLines == 0 || totalLines < config.MaxLines) {
remainingLines := config.MaxLines - totalLines
if remainingLines > 0 || config.MaxLines == 0 {
fileConfig := config
if config.MaxLines > 0 {
fileConfig.MaxLines = remainingLines
}
fileLines, err := olp.streamLogFileOptimized(currentLog, fileConfig)
if err != nil {
getLogger().WithError(err).WithField("file", currentLog).Error("Failed to read current log file")
} else {
lines = append(lines, fileLines...)
}
}
}
// Return a copy since we're pooling the original
result := make([]string, len(lines))
copy(result, lines)
return result, nil
}
// getCachedGlobResults caches glob results for performance
func (olp *OptimizedLogProcessor) getCachedGlobResults(pattern string) ([]string, error) {
// For now, don't cache glob results as file lists change frequently
// In a production system, you might cache with a TTL
return filepath.Glob(pattern)
}
// parseLogFilesOptimized optimizes file parsing with caching and better sorting
func (olp *OptimizedLogProcessor) parseLogFilesOptimized(files []string) (string, []OptimizedRotatedLog) {
var currentLog string
rotated := make([]OptimizedRotatedLog, 0, len(files))
for _, path := range files {
base := filepath.Base(path)
if base == "fail2ban.log" {
currentLog = path
} else if strings.HasPrefix(base, "fail2ban.log.") {
// Extract number more efficiently
if num := olp.extractLogNumberOptimized(base); num >= 0 {
info := olp.getCachedFileInfo(path)
rotated = append(rotated, OptimizedRotatedLog{
Num: num,
Path: path,
Info: info,
})
}
}
}
// Sort with cached info for better performance
olp.sortRotatedLogsOptimized(rotated)
return currentLog, rotated
}
// extractLogNumberOptimized efficiently extracts log numbers from filenames
func (olp *OptimizedLogProcessor) extractLogNumberOptimized(basename string) int {
// For "fail2ban.log.1" or "fail2ban.log.1.gz"
parts := strings.Split(basename, ".")
if len(parts) < 3 {
return -1
}
// parts[2] should be the number
numStr := parts[2]
if num, err := strconv.Atoi(numStr); err == nil && num >= 0 {
return num
}
return -1
}
// getCachedFileInfo gets or creates cached file information
func (olp *OptimizedLogProcessor) getCachedFileInfo(path string) *CachedFileInfo {
if cached, ok := olp.fileInfoCache.Load(path); ok {
olp.cacheHits.Add(1)
return cached.(*CachedFileInfo)
}
olp.cacheMisses.Add(1)
// Create new file info
info := &CachedFileInfo{
Path: path,
LogNumber: olp.extractLogNumberOptimized(filepath.Base(path)),
IsValid: true,
}
// Check if file is gzip
info.IsGzip = olp.isGzipFileOptimized(path)
// Get file size and mod time if needed for sorting
if stat, err := os.Stat(path); err == nil {
info.Size = stat.Size()
info.ModTime = stat.ModTime().Unix()
}
olp.fileInfoCache.Store(path, info)
return info
}
// isGzipFileOptimized provides cached gzip detection
func (olp *OptimizedLogProcessor) isGzipFileOptimized(path string) bool {
if cached, ok := olp.gzipCache.Load(path); ok {
return cached.(bool)
}
// Use optimized detection
isGzip := olp.fastGzipDetection(path)
olp.gzipCache.Store(path, isGzip)
return isGzip
}
// fastGzipDetection provides faster gzip detection
func (olp *OptimizedLogProcessor) fastGzipDetection(path string) bool {
// Super fast path: check extension
if strings.HasSuffix(path, ".gz") {
return true
}
// For fail2ban logs, if it doesn't end in .gz, it's very likely not gzipped
// We can skip the expensive magic byte check for known patterns
basename := filepath.Base(path)
if strings.HasPrefix(basename, "fail2ban.log") && !strings.Contains(basename, ".gz") {
return false
}
// Fallback to default detection only if necessary
isGzip, err := IsGzipFile(path)
if err != nil {
return false
}
return isGzip
}
// sortRotatedLogsOptimized provides optimized sorting
func (olp *OptimizedLogProcessor) sortRotatedLogsOptimized(rotated []OptimizedRotatedLog) {
// Use a more efficient sorting approach
sort.Slice(rotated, func(i, j int) bool {
// Primary sort: by log number (higher number = older)
if rotated[i].Num != rotated[j].Num {
return rotated[i].Num > rotated[j].Num
}
// Secondary sort: by modification time if numbers are equal
if rotated[i].Info != nil && rotated[j].Info != nil {
return rotated[i].Info.ModTime > rotated[j].Info.ModTime
}
// Fallback: string comparison
return rotated[i].Path > rotated[j].Path
})
}
// streamLogFileOptimized provides optimized log file streaming
func (olp *OptimizedLogProcessor) streamLogFileOptimized(path string, config LogReadConfig) ([]string, error) {
cleanPath, err := validateLogPath(path)
if err != nil {
return nil, err
}
if shouldSkipFile(cleanPath, config.MaxFileSize) {
return []string{}, nil
}
// Use cached gzip detection
isGzip := olp.isGzipFileOptimized(cleanPath)
// Create optimized scanner
scanner, cleanup, err := olp.createOptimizedScanner(cleanPath, isGzip)
if err != nil {
return nil, err
}
defer cleanup()
return olp.scanLogLinesOptimized(scanner, config)
}
// createOptimizedScanner creates an optimized scanner with pooled buffers
func (olp *OptimizedLogProcessor) createOptimizedScanner(path string, isGzip bool) (*bufio.Scanner, func(), error) {
if isGzip {
// Use existing gzip-aware scanner
return CreateGzipAwareScannerWithBuffer(path, 64*1024)
}
// For regular files, use optimized approach
// #nosec G304 - path is validated by validateLogPath before this call
file, err := os.Open(path)
if err != nil {
return nil, nil, err
}
// Get pooled buffer
bufPtr := olp.scannerPool.Get().(*[]byte)
buf := (*bufPtr)[:cap(*bufPtr)] // Use full capacity
scanner := bufio.NewScanner(file)
scanner.Buffer(buf, 64*1024) // 64KB max line size
cleanup := func() {
if err := file.Close(); err != nil {
getLogger().WithError(err).WithField("file", path).Warn("Failed to close file during cleanup")
}
*bufPtr = (*bufPtr)[:0] // Reset buffer
olp.scannerPool.Put(bufPtr)
}
return scanner, cleanup, nil
}
// scanLogLinesOptimized provides optimized line scanning with reduced allocations
func (olp *OptimizedLogProcessor) scanLogLinesOptimized(
scanner *bufio.Scanner,
config LogReadConfig,
) ([]string, error) {
// Get pooled string slice
linesPtr := olp.stringPool.Get().(*[]string)
lines := (*linesPtr)[:0] // Reset slice but keep capacity
defer func() {
*linesPtr = lines[:0]
olp.stringPool.Put(linesPtr)
}()
lineCount := 0
hasJailFilter := config.JailFilter != "" && config.JailFilter != "all"
hasIPFilter := config.IPFilter != "" && config.IPFilter != "all"
for scanner.Scan() {
if config.MaxLines > 0 && lineCount >= config.MaxLines {
break
}
line := scanner.Text()
if len(line) == 0 {
continue
}
// Fast filtering without trimming unless necessary
if hasJailFilter || hasIPFilter {
if !olp.matchesFiltersOptimized(line, config.JailFilter, config.IPFilter, hasJailFilter, hasIPFilter) {
continue
}
}
lines = append(lines, line)
lineCount++
}
if err := scanner.Err(); err != nil {
return nil, err
}
// Return a copy since we're pooling the original
result := make([]string, len(lines))
copy(result, lines)
return result, nil
}
// matchesFiltersOptimized provides optimized filtering with minimal allocations
func (olp *OptimizedLogProcessor) matchesFiltersOptimized(
line, jailFilter, ipFilter string,
hasJailFilter, hasIPFilter bool,
) bool {
if !hasJailFilter && !hasIPFilter {
return true
}
// Fast byte-level searching to avoid string allocations
lineBytes := []byte(line)
jailMatch := !hasJailFilter
ipMatch := !hasIPFilter
if hasJailFilter && !jailMatch {
// Look for jail pattern: [jail-name]
jailPattern := "[" + jailFilter + "]"
if olp.fastContains(lineBytes, []byte(jailPattern)) {
jailMatch = true
}
}
if hasIPFilter && !ipMatch {
// Look for IP pattern in the line
if olp.fastContains(lineBytes, []byte(ipFilter)) {
ipMatch = true
}
}
return jailMatch && ipMatch
}
// fastContains provides fast byte-level substring search
func (olp *OptimizedLogProcessor) fastContains(haystack, needle []byte) bool {
if len(needle) == 0 {
return true
}
if len(needle) > len(haystack) {
return false
}
// Use Boyer-Moore-like approach for longer needles
if len(needle) > 4 {
return strings.Contains(string(haystack), string(needle))
}
// Simple search for short needles
for i := 0; i <= len(haystack)-len(needle); i++ {
match := true
for j := 0; j < len(needle); j++ {
if haystack[i+j] != needle[j] {
match = false
break
}
}
if match {
return true
}
}
return false
}
// GetCacheStats returns cache performance statistics
func (olp *OptimizedLogProcessor) GetCacheStats() (hits, misses int64) {
return olp.cacheHits.Load(), olp.cacheMisses.Load()
}
// ClearCaches clears all caches (useful for testing or memory management)
func (olp *OptimizedLogProcessor) ClearCaches() {
// Use sync.Map's Range and Delete methods for thread-safe clearing
olp.gzipCache.Range(func(key, _ interface{}) bool {
olp.gzipCache.Delete(key)
return true
})
olp.pathCache.Range(func(key, _ interface{}) bool {
olp.pathCache.Delete(key)
return true
})
olp.fileInfoCache.Range(func(key, _ interface{}) bool {
olp.fileInfoCache.Delete(key)
return true
})
olp.cacheHits.Store(0)
olp.cacheMisses.Store(0)
}
// Global optimized processor instance
var optimizedLogProcessor = NewOptimizedLogProcessor()
// GetLogLinesUltraOptimized provides ultra-optimized log line retrieval
func GetLogLinesUltraOptimized(jailFilter, ipFilter string, maxLines int) ([]string, error) {
return optimizedLogProcessor.GetLogLinesOptimized(jailFilter, ipFilter, maxLines)
}

89
fail2ban/logging_context.go Normal file
View File

@@ -0,0 +1,89 @@
// Package fail2ban provides context utility functions for structured logging and tracing.
// This module handles context value management, logger creation with context fields,
// and request ID generation for better traceability in fail2ban operations.
package fail2ban
import (
"context"
"net"
"strings"
"github.com/google/uuid"
"github.com/ivuorinen/f2b/shared"
)
// WithRequestID adds a request ID to the context
func WithRequestID(ctx context.Context, requestID string) context.Context {
// Trim whitespace and validate
requestID = strings.TrimSpace(requestID)
if requestID == "" {
return ctx // Don't store empty request IDs
}
return context.WithValue(ctx, shared.ContextKeyRequestID, requestID)
}
// WithOperation adds an operation name to the context
func WithOperation(ctx context.Context, operation string) context.Context {
// Trim whitespace and validate
operation = strings.TrimSpace(operation)
if operation == "" {
return ctx // Don't store empty operations
}
return context.WithValue(ctx, shared.ContextKeyOperation, operation)
}
// WithJail adds a validated jail name to the context
func WithJail(ctx context.Context, jail string) context.Context {
jail = strings.TrimSpace(jail)
// Validate jail name before storing
if err := ValidateJail(jail); err != nil {
// Don't store invalid jail names in context
getLogger().WithError(err).Warn("Invalid jail name not stored in context")
return ctx
}
return context.WithValue(ctx, shared.ContextKeyJail, jail)
}
// WithIP adds a validated IP address to the context
func WithIP(ctx context.Context, ip string) context.Context {
ip = strings.TrimSpace(ip)
// Validate IP before storing
if net.ParseIP(ip) == nil {
getLogger().WithField("ip", ip).Warn("Invalid IP not stored in context")
return ctx
}
return context.WithValue(ctx, shared.ContextKeyIP, ip)
}
// LoggerFromContext creates a logger entry with fields from context
func LoggerFromContext(ctx context.Context) LoggerEntry {
fields := Fields{}
if requestID, ok := ctx.Value(shared.ContextKeyRequestID).(string); ok && requestID != "" {
fields["request_id"] = requestID
}
if operation, ok := ctx.Value(shared.ContextKeyOperation).(string); ok && operation != "" {
fields["operation"] = operation
}
if jail, ok := ctx.Value(shared.ContextKeyJail).(string); ok && jail != "" {
fields["jail"] = jail
}
if ip, ok := ctx.Value(shared.ContextKeyIP).(string); ok && ip != "" {
fields["ip"] = ip
}
return getLogger().WithFields(fields)
}
// GenerateRequestID generates a unique request ID using UUID for tracing
func GenerateRequestID() string {
return uuid.NewString()
}

90
fail2ban/logging_env.go Normal file
View File

@@ -0,0 +1,90 @@
// Package fail2ban provides logging and environment detection utilities.
// This module handles logger configuration, CI detection, and test environment setup
// for the fail2ban integration system.
package fail2ban
import (
"os"
"strings"
"sync/atomic"
"github.com/sirupsen/logrus"
)
// logger holds the current logger instance in a thread-safe manner
var logger atomic.Value
func init() {
// Initialize with default logger
logger.Store(NewLogrusAdapter(logrus.StandardLogger()))
}
// SetLogger allows the cmd package to set the logger instance (thread-safe)
func SetLogger(l LoggerInterface) {
if l == nil {
return
}
logger.Store(l)
}
// getLogger returns the current logger instance (thread-safe)
func getLogger() LoggerInterface {
l, ok := logger.Load().(LoggerInterface)
if !ok {
// Fallback to default logger if type assertion fails
return NewLogrusAdapter(logrus.StandardLogger())
}
return l
}
// IsCI detects if we're running in a CI environment
func IsCI() bool {
ciEnvVars := []string{
"CI", "GITHUB_ACTIONS", "TRAVIS", "CIRCLECI", "JENKINS_URL",
"BUILDKITE", "TF_BUILD", "GITLAB_CI",
}
for _, envVar := range ciEnvVars {
if os.Getenv(envVar) != "" {
return true
}
}
return false
}
// ConfigureCITestLogging reduces log verbosity in CI and test environments
// This should be called explicitly during application initialization
func ConfigureCITestLogging() {
if IsCI() || IsTestEnvironment() {
// Try interface-based assertion first to support custom loggers
currentLogger := getLogger()
if l, ok := currentLogger.(interface{ SetLevel(logrus.Level) }); ok {
l.SetLevel(logrus.WarnLevel)
} else {
// Log when we can't adjust level (observable for debugging)
logrus.StandardLogger().Debug(
"Non-standard logger in use; CI/test log level adjustment skipped",
)
}
}
}
// IsTestEnvironment detects if we're running in a test environment
func IsTestEnvironment() bool {
// Check for test-specific environment variables
testEnvVars := []string{"GO_TEST", "F2B_TEST", "F2B_TEST_SUDO"}
for _, envVar := range testEnvVars {
if os.Getenv(envVar) != "" {
return true
}
}
// Check command line arguments for test patterns
for _, arg := range os.Args {
if strings.Contains(arg, ".test") || strings.Contains(arg, "-test") {
return true
}
}
return false
}

237
fail2ban/logging_env_test.go Normal file
View File

@@ -0,0 +1,237 @@
package fail2ban
import (
"testing"
"github.com/sirupsen/logrus"
)
func TestSetLogger(t *testing.T) {
// Save original logger
originalLogger := getLogger()
defer SetLogger(originalLogger)
// Create a test logger
testLogger := NewLogrusAdapter(logrus.New())
// Set the logger
SetLogger(testLogger)
// Verify it was set
retrievedLogger := getLogger()
if retrievedLogger == nil {
t.Fatal("Retrieved logger is nil")
}
// Test that the logger is actually used
// We can't directly compare pointers, but we can verify it's not the original
if retrievedLogger == originalLogger {
t.Error("Logger was not updated")
}
}
func TestSetLogger_Concurrent(t *testing.T) {
// Save original logger
originalLogger := getLogger()
defer SetLogger(originalLogger)
// Test concurrent access to SetLogger and getLogger
done := make(chan bool)
for i := 0; i < 10; i++ {
go func() {
testLogger := NewLogrusAdapter(logrus.New())
SetLogger(testLogger)
_ = getLogger()
done <- true
}()
}
// Wait for all goroutines
for i := 0; i < 10; i++ {
<-done
}
// Verify we didn't panic and logger is set
if getLogger() == nil {
t.Error("Logger is nil after concurrent access")
}
}
func TestIsCI(t *testing.T) {
tests := []struct {
name string
envVars map[string]string
expected bool
}{
{
name: "GitHub Actions",
envVars: map[string]string{"GITHUB_ACTIONS": "true"},
expected: true,
},
{
name: "CI environment",
envVars: map[string]string{"CI": "true"},
expected: true,
},
{
name: "Travis CI",
envVars: map[string]string{"TRAVIS": "true"},
expected: true,
},
{
name: "CircleCI",
envVars: map[string]string{"CIRCLECI": "true"},
expected: true,
},
{
name: "Jenkins",
envVars: map[string]string{"JENKINS_URL": "http://jenkins"},
expected: true,
},
{
name: "GitLab CI",
envVars: map[string]string{"GITLAB_CI": "true"},
expected: true,
},
{
name: "No CI",
envVars: map[string]string{},
expected: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Clear all CI environment variables first using t.Setenv
ciVars := []string{
"CI",
"GITHUB_ACTIONS",
"TRAVIS",
"CIRCLECI",
"JENKINS_URL",
"BUILDKITE",
"TF_BUILD",
"GITLAB_CI",
}
for _, v := range ciVars {
t.Setenv(v, "")
}
// Set test environment variables using t.Setenv
for k, v := range tt.envVars {
t.Setenv(k, v)
}
result := IsCI()
if result != tt.expected {
t.Errorf("IsCI() = %v, want %v", result, tt.expected)
}
})
}
}
func TestIsTestEnvironment(t *testing.T) {
tests := []struct {
name string
envVars map[string]string
expected bool
}{
{
name: "GO_TEST set",
envVars: map[string]string{"GO_TEST": "true"},
expected: true,
},
{
name: "F2B_TEST set",
envVars: map[string]string{"F2B_TEST": "true"},
expected: true,
},
{
name: "F2B_TEST_SUDO set",
envVars: map[string]string{"F2B_TEST_SUDO": "true"},
expected: true,
},
{
name: "No test environment",
envVars: map[string]string{},
expected: true, // Will be true because we're running in test mode (os.Args contains -test)
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Clear test environment variables using t.Setenv
testVars := []string{"GO_TEST", "F2B_TEST", "F2B_TEST_SUDO"}
for _, v := range testVars {
t.Setenv(v, "")
}
// Set test environment variables using t.Setenv
for k, v := range tt.envVars {
t.Setenv(k, v)
}
result := IsTestEnvironment()
if result != tt.expected {
t.Errorf("IsTestEnvironment() = %v, want %v", result, tt.expected)
}
})
}
}
func TestConfigureCITestLogging(t *testing.T) {
// Save original logger
originalLogger := getLogger()
defer SetLogger(originalLogger)
tests := []struct {
name string
isCI bool
setup func(t *testing.T)
}{
{
name: "in CI environment",
isCI: true,
setup: func(t *testing.T) {
t.Helper()
t.Setenv("CI", "true")
},
},
{
name: "not in CI environment",
isCI: false,
setup: func(t *testing.T) {
t.Helper()
t.Setenv("CI", "")
t.Setenv("GITHUB_ACTIONS", "")
t.Setenv("TRAVIS", "")
t.Setenv("CIRCLECI", "")
t.Setenv("JENKINS_URL", "")
t.Setenv("BUILDKITE", "")
t.Setenv("TF_BUILD", "")
t.Setenv("GITLAB_CI", "")
t.Setenv("GO_TEST", "")
t.Setenv("F2B_TEST", "")
t.Setenv("F2B_TEST_SUDO", "")
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
tt.setup(t)
// Create a new logrus logger to test with
testLogrusLogger := logrus.New()
testLogger := NewLogrusAdapter(testLogrusLogger)
SetLogger(testLogger)
// Call ConfigureCITestLogging
ConfigureCITestLogging()
// The function should not panic - that's the main test
// We can't easily verify the log level was changed without accessing internal state
// but we can verify the function runs without error
})
}
}

139
fail2ban/logrus_adapter.go Normal file
View File

@@ -0,0 +1,139 @@
package fail2ban
import "github.com/sirupsen/logrus"
// logrusAdapter wraps logrus to implement our decoupled LoggerInterface
type logrusAdapter struct {
entry *logrus.Entry
}
// logrusEntryAdapter wraps logrus.Entry to implement LoggerEntry
type logrusEntryAdapter struct {
entry *logrus.Entry
}
// Ensure logrusAdapter implements LoggerInterface
var _ LoggerInterface = (*logrusAdapter)(nil)
// Ensure logrusEntryAdapter implements LoggerEntry
var _ LoggerEntry = (*logrusEntryAdapter)(nil)
// NewLogrusAdapter creates a logger adapter from a logrus logger
func NewLogrusAdapter(logger *logrus.Logger) LoggerInterface {
if logger == nil {
logger = logrus.StandardLogger()
}
return &logrusAdapter{entry: logrus.NewEntry(logger)}
}
// WithField implements LoggerInterface
func (l *logrusAdapter) WithField(key string, value interface{}) LoggerEntry {
return &logrusEntryAdapter{entry: l.entry.WithField(key, value)}
}
// WithFields implements LoggerInterface
func (l *logrusAdapter) WithFields(fields Fields) LoggerEntry {
return &logrusEntryAdapter{entry: l.entry.WithFields(logrus.Fields(fields))}
}
// WithError implements LoggerInterface
func (l *logrusAdapter) WithError(err error) LoggerEntry {
return &logrusEntryAdapter{entry: l.entry.WithError(err)}
}
// Debug implements LoggerInterface
func (l *logrusAdapter) Debug(args ...interface{}) {
l.entry.Debug(args...)
}
// Info implements LoggerInterface
func (l *logrusAdapter) Info(args ...interface{}) {
l.entry.Info(args...)
}
// Warn implements LoggerInterface
func (l *logrusAdapter) Warn(args ...interface{}) {
l.entry.Warn(args...)
}
// Error implements LoggerInterface
func (l *logrusAdapter) Error(args ...interface{}) {
l.entry.Error(args...)
}
// Debugf implements LoggerInterface
func (l *logrusAdapter) Debugf(format string, args ...interface{}) {
l.entry.Debugf(format, args...)
}
// Infof implements LoggerInterface
func (l *logrusAdapter) Infof(format string, args ...interface{}) {
l.entry.Infof(format, args...)
}
// Warnf implements LoggerInterface
func (l *logrusAdapter) Warnf(format string, args ...interface{}) {
l.entry.Warnf(format, args...)
}
// Errorf implements LoggerInterface
func (l *logrusAdapter) Errorf(format string, args ...interface{}) {
l.entry.Errorf(format, args...)
}
// LoggerEntry implementation for logrusEntryAdapter
// WithField implements LoggerEntry
func (e *logrusEntryAdapter) WithField(key string, value interface{}) LoggerEntry {
return &logrusEntryAdapter{entry: e.entry.WithField(key, value)}
}
// WithFields implements LoggerEntry
func (e *logrusEntryAdapter) WithFields(fields Fields) LoggerEntry {
return &logrusEntryAdapter{entry: e.entry.WithFields(logrus.Fields(fields))}
}
// WithError implements LoggerEntry
func (e *logrusEntryAdapter) WithError(err error) LoggerEntry {
return &logrusEntryAdapter{entry: e.entry.WithError(err)}
}
// Debug implements LoggerEntry
func (e *logrusEntryAdapter) Debug(args ...interface{}) {
e.entry.Debug(args...)
}
// Info implements LoggerEntry
func (e *logrusEntryAdapter) Info(args ...interface{}) {
e.entry.Info(args...)
}
// Warn implements LoggerEntry
func (e *logrusEntryAdapter) Warn(args ...interface{}) {
e.entry.Warn(args...)
}
// Error implements LoggerEntry
func (e *logrusEntryAdapter) Error(args ...interface{}) {
e.entry.Error(args...)
}
// Debugf implements LoggerEntry
func (e *logrusEntryAdapter) Debugf(format string, args ...interface{}) {
e.entry.Debugf(format, args...)
}
// Infof implements LoggerEntry
func (e *logrusEntryAdapter) Infof(format string, args ...interface{}) {
e.entry.Infof(format, args...)
}
// Warnf implements LoggerEntry
func (e *logrusEntryAdapter) Warnf(format string, args ...interface{}) {
e.entry.Warnf(format, args...)
}
// Errorf implements LoggerEntry
func (e *logrusEntryAdapter) Errorf(format string, args ...interface{}) {
e.entry.Errorf(format, args...)
}

303
fail2ban/logrus_adapter_test.go Normal file
View File

@@ -0,0 +1,303 @@
package fail2ban
import (
"bytes"
"encoding/json"
"errors"
"testing"
"github.com/sirupsen/logrus"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestLogrusAdapter_ImplementsInterface(_ *testing.T) {
logger := logrus.New()
adapter := NewLogrusAdapter(logger)
// Should implement LoggerInterface
var _ = adapter
}
func TestLogrusAdapter_WithField(t *testing.T) {
var buf bytes.Buffer
logger := logrus.New()
logger.SetOutput(&buf)
logger.SetFormatter(&logrus.JSONFormatter{})
logger.SetLevel(logrus.InfoLevel)
adapter := NewLogrusAdapter(logger)
entry := adapter.WithField("test", "value")
// Should return LoggerEntry
var _ = entry
entry.Info("test message")
output := buf.String()
assert.Contains(t, output, "test")
assert.Contains(t, output, "value")
assert.Contains(t, output, "test message")
}
func TestLogrusAdapter_WithFields(t *testing.T) {
var buf bytes.Buffer
logger := logrus.New()
logger.SetOutput(&buf)
logger.SetFormatter(&logrus.JSONFormatter{})
logger.SetLevel(logrus.InfoLevel)
adapter := NewLogrusAdapter(logger)
fields := Fields{
"field1": "value1",
"field2": 42,
}
entry := adapter.WithFields(fields)
entry.Info("multi-field message")
output := buf.String()
assert.Contains(t, output, "field1")
assert.Contains(t, output, "value1")
assert.Contains(t, output, "field2")
assert.Contains(t, output, "42")
}
func TestLogrusAdapter_WithError(t *testing.T) {
var buf bytes.Buffer
logger := logrus.New()
logger.SetOutput(&buf)
logger.SetFormatter(&logrus.JSONFormatter{})
logger.SetLevel(logrus.ErrorLevel)
adapter := NewLogrusAdapter(logger)
testErr := errors.New("test error")
entry := adapter.WithError(testErr)
entry.Error("error occurred")
output := buf.String()
assert.Contains(t, output, "test error")
assert.Contains(t, output, "error occurred")
}
func TestLogrusAdapter_Chaining(t *testing.T) {
var buf bytes.Buffer
logger := logrus.New()
logger.SetOutput(&buf)
logger.SetFormatter(&logrus.JSONFormatter{})
logger.SetLevel(logrus.InfoLevel)
adapter := NewLogrusAdapter(logger)
// Test method chaining
adapter.
WithField("field1", "value1").
WithField("field2", "value2").
WithError(errors.New("chain error")).
Info("chained message")
output := buf.String()
assert.Contains(t, output, "field1")
assert.Contains(t, output, "field2")
assert.Contains(t, output, "chain error")
assert.Contains(t, output, "chained message")
}
func TestLogrusAdapter_LogLevels(t *testing.T) {
tests := []struct {
name string
logLevel logrus.Level
logFunc func(LoggerInterface)
expected bool
}{
{
name: "debug_enabled",
logLevel: logrus.DebugLevel,
logFunc: func(l LoggerInterface) { l.Debug("debug message") },
expected: true,
},
{
name: "info_enabled",
logLevel: logrus.InfoLevel,
logFunc: func(l LoggerInterface) { l.Info("info message") },
expected: true,
},
{
name: "warn_enabled",
logLevel: logrus.WarnLevel,
logFunc: func(l LoggerInterface) { l.Warn("warn message") },
expected: true,
},
{
name: "error_enabled",
logLevel: logrus.ErrorLevel,
logFunc: func(l LoggerInterface) { l.Error("error message") },
expected: true,
},
{
name: "debug_disabled_at_info_level",
logLevel: logrus.InfoLevel,
logFunc: func(l LoggerInterface) { l.Debug("debug message") },
expected: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
var buf bytes.Buffer
logger := logrus.New()
logger.SetOutput(&buf)
logger.SetLevel(tt.logLevel)
adapter := NewLogrusAdapter(logger)
tt.logFunc(adapter)
output := buf.String()
if tt.expected {
assert.NotEmpty(t, output, "Expected log output")
} else {
assert.Empty(t, output, "Expected no log output")
}
})
}
}
func TestLogrusAdapter_FormattedLogs(t *testing.T) {
tests := []struct {
name string
logFunc func(LoggerInterface)
expected string
}{
{
name: "debugf",
logFunc: func(l LoggerInterface) { l.Debugf("formatted %s %d", "test", 42) },
expected: "formatted test 42",
},
{
name: "infof",
logFunc: func(l LoggerInterface) { l.Infof("info %s", "test") },
expected: "info test",
},
{
name: "warnf",
logFunc: func(l LoggerInterface) { l.Warnf("warn %d", 123) },
expected: "warn 123",
},
{
name: "errorf",
logFunc: func(l LoggerInterface) { l.Errorf("error %v", "failed") },
expected: "error failed",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
var buf bytes.Buffer
logger := logrus.New()
logger.SetOutput(&buf)
logger.SetLevel(logrus.DebugLevel)
adapter := NewLogrusAdapter(logger)
tt.logFunc(adapter)
output := buf.String()
assert.Contains(t, output, tt.expected)
})
}
}
func TestLogrusEntryAdapter_Chaining(t *testing.T) {
var buf bytes.Buffer
logger := logrus.New()
logger.SetOutput(&buf)
logger.SetFormatter(&logrus.JSONFormatter{})
logger.SetLevel(logrus.InfoLevel)
adapter := NewLogrusAdapter(logger)
// Test entry-level chaining
entry := adapter.WithField("initial", "value")
entry.
WithField("chained1", "val1").
WithField("chained2", "val2").
Info("entry chain test")
output := buf.String()
assert.Contains(t, output, "initial")
assert.Contains(t, output, "chained1")
assert.Contains(t, output, "chained2")
assert.Contains(t, output, "entry chain test")
}
func TestLogrusAdapter_JSONOutput(t *testing.T) {
var buf bytes.Buffer
logger := logrus.New()
logger.SetOutput(&buf)
logger.SetFormatter(&logrus.JSONFormatter{})
logger.SetLevel(logrus.InfoLevel)
adapter := NewLogrusAdapter(logger)
adapter.WithFields(Fields{
"service": "f2b",
"version": "1.0.0",
}).Info("structured log")
// Verify valid JSON output
var logEntry map[string]interface{}
err := json.Unmarshal(buf.Bytes(), &logEntry)
require.NoError(t, err, "Output should be valid JSON")
assert.Equal(t, "f2b", logEntry["service"])
assert.Equal(t, "1.0.0", logEntry["version"])
assert.Contains(t, logEntry["msg"], "structured log")
}
func TestLogrusEntryAdapter_FormattedLogs(t *testing.T) {
var buf bytes.Buffer
logger := logrus.New()
logger.SetOutput(&buf)
logger.SetLevel(logrus.DebugLevel)
adapter := NewLogrusAdapter(logger)
entry := adapter.WithField("context", "test")
// Test formatted log methods on entry
entry.Debugf("debug %s", "formatted")
assert.Contains(t, buf.String(), "debug formatted")
buf.Reset()
entry.Infof("info %d", 42)
assert.Contains(t, buf.String(), "info 42")
buf.Reset()
entry.Warnf("warn %v", true)
assert.Contains(t, buf.String(), "warn true")
buf.Reset()
entry.Errorf("error %s", "test")
assert.Contains(t, buf.String(), "error test")
}
func TestLogrusAdapter_MultipleAdapters(t *testing.T) {
// Test that multiple adapters can coexist
logger1 := logrus.New()
logger2 := logrus.New()
var buf1, buf2 bytes.Buffer
logger1.SetOutput(&buf1)
logger2.SetOutput(&buf2)
adapter1 := NewLogrusAdapter(logger1)
adapter2 := NewLogrusAdapter(logger2)
adapter1.Info("message 1")
adapter2.Info("message 2")
assert.Contains(t, buf1.String(), "message 1")
assert.NotContains(t, buf1.String(), "message 2")
assert.Contains(t, buf2.String(), "message 2")
assert.NotContains(t, buf2.String(), "message 1")
}

463
fail2ban/logs.go
View File

@@ -3,14 +3,17 @@ package fail2ban
import (
"bufio"
"context"
"errors"
"fmt"
"io"
"net/url"
"net"
"os"
"path/filepath"
"sort"
"strconv"
"strings"
"github.com/ivuorinen/f2b/shared"
)
/*
@@ -26,18 +29,63 @@ including support for rotated and compressed logs.
//
// Returns a slice of matching log lines, or an error.
// This function uses streaming to limit memory usage.
func GetLogLines(jailFilter string, ipFilter string) ([]string, error) {
return GetLogLinesWithLimit(jailFilter, ipFilter, 1000) // Default limit for safety
// Context parameter supports timeout and cancellation of file I/O operations.
func GetLogLines(ctx context.Context, jailFilter string, ipFilter string) ([]string, error) {
return GetLogLinesWithLimit(ctx, jailFilter, ipFilter, shared.DefaultLogLinesLimit) // Default limit for safety
}
// GetLogLinesWithLimit returns log lines with configurable limits for memory management.
func GetLogLinesWithLimit(jailFilter string, ipFilter string, maxLines int) ([]string, error) {
// Handle zero limit case - return empty slice immediately
// Context parameter supports timeout and cancellation of file I/O operations.
func GetLogLinesWithLimit(ctx context.Context, jailFilter string, ipFilter string, maxLines int) ([]string, error) {
// Validate maxLines parameter
if maxLines < 0 {
return nil, fmt.Errorf(shared.ErrMaxLinesNegative, maxLines)
}
if maxLines > shared.MaxLogLinesLimit {
return nil, fmt.Errorf(shared.ErrMaxLinesExceedsLimit, shared.MaxLogLinesLimit)
}
if maxLines == 0 {
return []string{}, nil
}
pattern := filepath.Join(GetLogDir(), "fail2ban.log*")
// Sanitize filter parameters
jailFilter = strings.TrimSpace(jailFilter)
ipFilter = strings.TrimSpace(ipFilter)
// Validate jail filter
if jailFilter != "" {
if err := ValidateJail(jailFilter); err != nil {
return nil, fmt.Errorf("invalid jail filter: %w", err)
}
}
// Validate IP filter
if ipFilter != "" && ipFilter != shared.AllFilter {
if net.ParseIP(ipFilter) == nil {
return nil, fmt.Errorf(shared.ErrInvalidIPAddress, ipFilter)
}
}
config := LogReadConfig{
MaxLines: maxLines,
MaxFileSize: shared.DefaultMaxFileSize,
JailFilter: jailFilter,
IPFilter: ipFilter,
BaseDir: GetLogDir(),
}
return collectLogLines(ctx, GetLogDir(), config)
}
// collectLogLines reads log files under the provided directory using the supplied configuration.
func collectLogLines(ctx context.Context, logDir string, baseConfig LogReadConfig) ([]string, error) {
if baseConfig.MaxLines == 0 {
return []string{}, nil
}
pattern := filepath.Join(logDir, "fail2ban.log*")
files, err := filepath.Glob(pattern)
if err != nil {
return nil, fmt.Errorf("error listing log files: %w", err)
@@ -49,66 +97,59 @@ func GetLogLinesWithLimit(jailFilter string, ipFilter string, maxLines int) ([]s
currentLog, rotated := parseLogFiles(files)
// Use streaming approach with memory limits
config := LogReadConfig{
MaxLines: maxLines,
MaxFileSize: 100 * 1024 * 1024, // 100MB file size limit
JailFilter: jailFilter,
IPFilter: ipFilter,
ReverseOrder: false,
var allLines []string
appendAndTrim := func(lines []string) {
if len(lines) == 0 {
return
}
allLines = append(allLines, lines...)
if baseConfig.MaxLines > 0 && len(allLines) > baseConfig.MaxLines {
allLines = allLines[len(allLines)-baseConfig.MaxLines:]
}
}
var allLines []string
totalLines := 0
// Read rotated logs first (oldest to newest) - maintains original ordering
for _, rotatedFile := range rotated {
if config.MaxLines > 0 && totalLines >= config.MaxLines {
break
}
// Adjust remaining lines limit (skip limit check for negative MaxLines)
fileConfig := config
if config.MaxLines > 0 {
remainingLines := config.MaxLines - totalLines
if remainingLines <= 0 {
break
}
fileConfig.MaxLines = remainingLines
}
lines, err := streamLogFile(rotatedFile.path, fileConfig)
fileLines, err := readLogLinesFromFile(ctx, rotatedFile.path, baseConfig)
if err != nil {
getLogger().WithError(err).WithField("file", rotatedFile.path).Error("Failed to read rotated log file")
if ctx != nil && errors.Is(err, ctx.Err()) {
return nil, err
}
getLogger().WithError(err).
WithField(shared.LogFieldFile, rotatedFile.path).
Error("Failed to read rotated log file")
continue
}
allLines = append(allLines, lines...)
totalLines += len(lines)
appendAndTrim(fileLines)
}
// Read current log last (most recent) - maintains original ordering
if currentLog != "" && (config.MaxLines <= 0 || totalLines < config.MaxLines) {
fileConfig := config
if config.MaxLines > 0 {
remainingLines := config.MaxLines - totalLines
if remainingLines <= 0 {
return allLines, nil
}
fileConfig.MaxLines = remainingLines
}
lines, err := streamLogFile(currentLog, fileConfig)
if currentLog != "" {
fileLines, err := readLogLinesFromFile(ctx, currentLog, baseConfig)
if err != nil {
getLogger().WithError(err).WithField("file", currentLog).Error("Failed to read current log file")
if ctx != nil && errors.Is(err, ctx.Err()) {
return nil, err
}
getLogger().WithError(err).
WithField(shared.LogFieldFile, currentLog).
Error("Failed to read current log file")
} else {
allLines = append(allLines, lines...)
appendAndTrim(fileLines)
}
}
return allLines, nil
}
func readLogLinesFromFile(ctx context.Context, path string, baseConfig LogReadConfig) ([]string, error) {
fileConfig := baseConfig
fileConfig.MaxLines = 0
if ctx != nil {
return streamLogFileWithContext(ctx, path, fileConfig)
}
return streamLogFile(path, fileConfig)
}
// parseLogFiles parses log file names and returns the current log and a slice of rotated logs
// (sorted oldest to newest).
func parseLogFiles(files []string) (string, []rotatedLog) {
@@ -117,9 +158,9 @@ func parseLogFiles(files []string) (string, []rotatedLog) {
for _, path := range files {
base := filepath.Base(path)
if base == "fail2ban.log" {
if base == shared.LogFileName {
currentLog = path
} else if strings.HasPrefix(base, "fail2ban.log.") {
} else if strings.HasPrefix(base, shared.LogFilePrefix) {
if num := extractLogNumber(base); num >= 0 {
rotated = append(rotated, rotatedLog{num: num, path: path})
}
@@ -137,7 +178,7 @@ func parseLogFiles(files []string) (string, []rotatedLog) {
// extractLogNumber extracts the rotation number from a log file name (e.g., "fail2ban.log.2.gz" -> 2).
func extractLogNumber(base string) int {
numPart := strings.TrimPrefix(base, "fail2ban.log.")
numPart = strings.TrimSuffix(numPart, ".gz")
numPart = strings.TrimSuffix(numPart, shared.GzipExtension)
if n, err := strconv.Atoi(numPart); err == nil {
return n
}
@@ -152,31 +193,24 @@ type rotatedLog struct {
// LogReadConfig holds configuration for streaming log reading
type LogReadConfig struct {
MaxLines int // Maximum number of lines to read (0 = unlimited)
MaxFileSize int64 // Maximum file size to process in bytes (0 = unlimited)
JailFilter string // Filter by jail name (empty = no filter)
IPFilter string // Filter by IP address (empty = no filter)
ReverseOrder bool // Read from end of file backwards (for recent logs)
MaxLines int // Maximum number of lines to read (0 = unlimited)
MaxFileSize int64 // Maximum file size to process in bytes (0 = unlimited)
JailFilter string // Filter by jail name (empty = no filter)
IPFilter string // Filter by IP address (empty = no filter)
BaseDir string // Base directory for log validation
}
// resolveBaseDir returns the base directory from config or falls back to GetLogDir()
func resolveBaseDir(config LogReadConfig) string {
if config.BaseDir != "" {
return config.BaseDir
}
return GetLogDir()
}
// streamLogFile reads a log file line by line with memory limits and filtering
func streamLogFile(path string, config LogReadConfig) ([]string, error) {
cleanPath, err := validateLogPath(path)
if err != nil {
return nil, err
}
if shouldSkipFile(cleanPath, config.MaxFileSize) {
return []string{}, nil
}
scanner, cleanup, err := createLogScanner(cleanPath)
if err != nil {
return nil, err
}
defer cleanup()
return scanLogLines(scanner, config)
return streamLogFileWithContext(context.Background(), path, config)
}
// streamLogFileWithContext reads a log file line by line with memory limits,
@@ -189,7 +223,8 @@ func streamLogFileWithContext(ctx context.Context, path string, config LogReadCo
default:
}
cleanPath, err := validateLogPath(path)
baseDir := resolveBaseDir(config)
cleanPath, err := validateLogPathForDir(ctx, path, baseDir)
if err != nil {
return nil, err
}
@@ -207,218 +242,13 @@ func streamLogFileWithContext(ctx context.Context, path string, config LogReadCo
return scanLogLinesWithContext(ctx, scanner, config)
}
// PathSecurityConfig holds configuration for path security validation
type PathSecurityConfig struct {
AllowedBasePaths []string // List of allowed base directories
MaxPathLength int // Maximum allowed path length (0 = unlimited)
AllowSymlinks bool // Whether to allow symlinks
ResolveSymlinks bool // Whether to resolve symlinks before validation
}
// validateLogPath validates and sanitizes the log file path with comprehensive security checks
func validateLogPath(path string) (string, error) {
config := PathSecurityConfig{
AllowedBasePaths: []string{GetLogDir()}, // Use configured log directory
MaxPathLength: 4096, // Reasonable path length limit
AllowSymlinks: false, // Disable symlinks for security
ResolveSymlinks: true, // Resolve symlinks before validation
}
return validatePathWithSecurity(path, config)
return validateLogPathForDir(context.Background(), path, GetLogDir())
}
// validatePathWithSecurity performs comprehensive path security validation
func validatePathWithSecurity(path string, config PathSecurityConfig) (string, error) {
if path == "" {
return "", fmt.Errorf("empty path not allowed")
}
// Check path length limits
if config.MaxPathLength > 0 && len(path) > config.MaxPathLength {
return "", fmt.Errorf("path too long: %d characters (max: %d)", len(path), config.MaxPathLength)
}
// Detect and prevent null byte injection
if strings.Contains(path, "\x00") {
return "", fmt.Errorf("path contains null byte")
}
// Decode URL-encoded path traversal attempts
if decodedPath, err := url.QueryUnescape(path); err == nil && decodedPath != path {
getLogger().WithField("original", path).WithField("decoded", decodedPath).
Warn("Detected URL-encoded path, using decoded version for validation")
path = decodedPath
}
// Normalize unicode characters to prevent bypass attempts
path = normalizeUnicode(path)
// Basic path traversal detection (before cleaning)
if hasPathTraversal(path) {
return "", fmt.Errorf("path contains path traversal patterns")
}
// Clean and resolve the path
cleanPath, err := filepath.Abs(filepath.Clean(path))
if err != nil {
return "", fmt.Errorf("invalid path: %w", err)
}
// Additional check after cleaning (double-check for sophisticated attacks)
if hasPathTraversal(cleanPath) {
return "", fmt.Errorf("path contains path traversal patterns after normalization")
}
// Handle symlinks according to configuration
finalPath, err := handleSymlinks(cleanPath, config)
if err != nil {
return "", err
}
// Validate against allowed base paths
if err := validateBasePath(finalPath, config.AllowedBasePaths); err != nil {
return "", err
}
// Check if path points to a device file or other dangerous file types
if err := validateFileType(finalPath); err != nil {
return "", err
}
return finalPath, nil
}
// hasPathTraversal detects various path traversal patterns
func hasPathTraversal(path string) bool {
// Check for various path traversal patterns
dangerousPatterns := []string{
"..",
"./",
".\\",
"//",
"\\\\",
"/../",
"\\..\\",
"%2e%2e", // URL encoded ..
"%2f", // URL encoded /
"%5c", // URL encoded \
"\u002e\u002e", // Unicode ..
"\u2024\u2024", // Unicode bullet points (can look like ..)
"\uff0e\uff0e", // Full-width Unicode ..
}
pathLower := strings.ToLower(path)
for _, pattern := range dangerousPatterns {
if strings.Contains(pathLower, strings.ToLower(pattern)) {
return true
}
}
return false
}
// normalizeUnicode normalizes unicode characters to prevent bypass attempts
func normalizeUnicode(path string) string {
// Replace various Unicode representations of dots and slashes
replacements := map[string]string{
"\u002e": ".", // Unicode dot
"\u2024": ".", // Unicode bullet (one dot leader)
"\uff0e": ".", // Full-width dot
"\u002f": "/", // Unicode slash
"\u2044": "/", // Unicode fraction slash
"\uff0f": "/", // Full-width slash
"\u005c": "\\", // Unicode backslash
"\uff3c": "\\", // Full-width backslash
}
result := path
for unicode, ascii := range replacements {
result = strings.ReplaceAll(result, unicode, ascii)
}
return result
}
// handleSymlinks resolves or validates symlinks according to configuration
func handleSymlinks(path string, config PathSecurityConfig) (string, error) {
// Check if the path is a symlink
if info, err := os.Lstat(path); err == nil {
if info.Mode()&os.ModeSymlink != 0 {
if !config.AllowSymlinks {
return "", fmt.Errorf("symlinks not allowed: %s", path)
}
if config.ResolveSymlinks {
resolved, err := filepath.EvalSymlinks(path)
if err != nil {
return "", fmt.Errorf("failed to resolve symlink: %w", err)
}
return resolved, nil
}
}
} else if !os.IsNotExist(err) {
return "", fmt.Errorf("failed to check file info: %w", err)
}
return path, nil
}
// validateBasePath ensures the path is within allowed base directories
func validateBasePath(path string, allowedBasePaths []string) error {
if len(allowedBasePaths) == 0 {
return nil // No restrictions if no base paths configured
}
for _, basePath := range allowedBasePaths {
cleanBasePath, err := filepath.Abs(filepath.Clean(basePath))
if err != nil {
continue
}
// Check if path starts with allowed base path
if strings.HasPrefix(path, cleanBasePath+string(filepath.Separator)) ||
path == cleanBasePath {
return nil
}
}
return fmt.Errorf("path outside allowed directories: %s", path)
}
// validateFileType checks for dangerous file types (devices, named pipes, etc.)
func validateFileType(path string) error {
// Check if file exists
info, err := os.Stat(path)
if os.IsNotExist(err) {
return nil // File doesn't exist yet, allow it
}
if err != nil {
return fmt.Errorf("failed to stat file: %w", err)
}
mode := info.Mode()
// Block device files
if mode&os.ModeDevice != 0 {
return fmt.Errorf("device files not allowed: %s", path)
}
// Block named pipes (FIFOs)
if mode&os.ModeNamedPipe != 0 {
return fmt.Errorf("named pipes not allowed: %s", path)
}
// Block socket files
if mode&os.ModeSocket != 0 {
return fmt.Errorf("socket files not allowed: %s", path)
}
// Block irregular files (anything that's not a regular file or directory)
if !mode.IsRegular() && !mode.IsDir() {
return fmt.Errorf("irregular file type not allowed: %s", path)
}
return nil
func validateLogPathForDir(ctx context.Context, path string, baseDir string) (string, error) {
return ValidateLogPath(ctx, path, baseDir)
}
// shouldSkipFile checks if a file should be skipped due to size limits
@@ -429,7 +259,7 @@ func shouldSkipFile(path string, maxFileSize int64) bool {
if info, err := os.Stat(path); err == nil {
if info.Size() > maxFileSize {
getLogger().WithField("file", path).WithField("size", info.Size()).
getLogger().WithField(shared.LogFieldFile, path).WithField("size", info.Size()).
Warn("Skipping large log file due to size limit")
return true
}
@@ -468,7 +298,7 @@ func scanLogLines(scanner *bufio.Scanner, config LogReadConfig) ([]string, error
}
if err := scanner.Err(); err != nil {
return nil, fmt.Errorf("error scanning log file: %w", err)
return nil, fmt.Errorf(shared.ErrScanLogFile, err)
}
return lines, nil
@@ -509,7 +339,7 @@ func scanLogLinesWithContext(ctx context.Context, scanner *bufio.Scanner, config
}
if err := scanner.Err(); err != nil {
return nil, fmt.Errorf("error scanning log file: %w", err)
return nil, fmt.Errorf(shared.ErrScanLogFile, err)
}
return lines, nil
@@ -517,14 +347,14 @@ func scanLogLinesWithContext(ctx context.Context, scanner *bufio.Scanner, config
// passesFilters checks if a log line passes the configured filters
func passesFilters(line string, config LogReadConfig) bool {
if config.JailFilter != "" && config.JailFilter != AllFilter {
if config.JailFilter != "" && config.JailFilter != shared.AllFilter {
jailPattern := fmt.Sprintf("[%s]", config.JailFilter)
if !strings.Contains(line, jailPattern) {
return false
}
}
if config.IPFilter != "" && config.IPFilter != AllFilter {
if config.IPFilter != "" && config.IPFilter != shared.AllFilter {
if !strings.Contains(line, config.IPFilter) {
return false
}
@@ -555,3 +385,60 @@ func readLogFile(path string) ([]byte, error) {
return io.ReadAll(reader)
}
// OptimizedLogProcessor is a thin wrapper maintained for backwards compatibility
// with existing benchmarks and tests. Internally it delegates to the shared log collection
// helpers so we have a single codepath to maintain.
type OptimizedLogProcessor struct{}
// NewOptimizedLogProcessor creates a new optimized processor wrapper.
func NewOptimizedLogProcessor() *OptimizedLogProcessor {
return &OptimizedLogProcessor{}
}
// GetLogLinesOptimized proxies to the shared collector to keep behavior identical
// while allowing benchmarks to exercise this entrypoint.
func (olp *OptimizedLogProcessor) GetLogLinesOptimized(jailFilter, ipFilter string, maxLines int) ([]string, error) {
// Validate maxLines parameter
if maxLines < 0 {
return nil, fmt.Errorf(shared.ErrMaxLinesNegative, maxLines)
}
if maxLines > shared.MaxLogLinesLimit {
return nil, fmt.Errorf(shared.ErrMaxLinesExceedsLimit, shared.MaxLogLinesLimit)
}
// Sanitize filter parameters
jailFilter = strings.TrimSpace(jailFilter)
ipFilter = strings.TrimSpace(ipFilter)
config := LogReadConfig{
MaxLines: maxLines,
MaxFileSize: shared.DefaultMaxFileSize,
JailFilter: jailFilter,
IPFilter: ipFilter,
BaseDir: GetLogDir(),
}
return collectLogLines(context.Background(), GetLogDir(), config)
}
// GetCacheStats is a no-op maintained for test compatibility.
// No caching is actually performed by this processor.
func (olp *OptimizedLogProcessor) GetCacheStats() (hits, misses int64) {
return 0, 0
}
// ClearCaches is a no-op maintained for test compatibility.
// No caching is actually performed by this processor.
func (olp *OptimizedLogProcessor) ClearCaches() {
// No-op: no cache state to clear
}
var optimizedLogProcessor = NewOptimizedLogProcessor()
// GetLogLinesUltraOptimized retains the legacy API that benchmarks expect while now
// sharing the simplified implementation.
func GetLogLinesUltraOptimized(jailFilter, ipFilter string, maxLines int) ([]string, error) {
return optimizedLogProcessor.GetLogLinesOptimized(jailFilter, ipFilter, maxLines)
}

380
fail2ban/logs_additional_test.go Normal file
View File

@@ -0,0 +1,380 @@
package fail2ban
import (
"bufio"
"context"
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// TestStreamLogFile tests the streamLogFile function
func TestStreamLogFile(t *testing.T) {
tmpDir := t.TempDir()
logFile := filepath.Join(tmpDir, "test.log")
logContent := `2024-01-01 10:00:00 [sshd] Ban 192.168.1.1
2024-01-01 10:01:00 [sshd] Ban 192.168.1.2
2024-01-01 10:02:00 [apache] Ban 192.168.1.3
`
err := os.WriteFile(logFile, []byte(logContent), 0600)
require.NoError(t, err)
t.Run("successful stream", func(t *testing.T) {
config := LogReadConfig{
MaxLines: 10,
BaseDir: tmpDir,
}
lines, err := streamLogFile(logFile, config)
assert.NoError(t, err)
assert.Len(t, lines, 3)
})
t.Run("stream with max lines limit", func(t *testing.T) {
config := LogReadConfig{
MaxLines: 2,
BaseDir: tmpDir,
}
lines, err := streamLogFile(logFile, config)
assert.NoError(t, err)
assert.LessOrEqual(t, len(lines), 2)
})
t.Run("stream with jail filter", func(t *testing.T) {
config := LogReadConfig{
MaxLines: 10,
JailFilter: "sshd",
BaseDir: tmpDir,
}
lines, err := streamLogFile(logFile, config)
assert.NoError(t, err)
for _, line := range lines {
assert.Contains(t, line, "sshd")
}
})
t.Run("stream with IP filter", func(t *testing.T) {
config := LogReadConfig{
MaxLines: 10,
IPFilter: "192.168.1.1",
BaseDir: tmpDir,
}
lines, err := streamLogFile(logFile, config)
assert.NoError(t, err)
for _, line := range lines {
assert.Contains(t, line, "192.168.1.1")
}
})
}
// TestScanLogLines tests the scanLogLines function
func TestScanLogLines(t *testing.T) {
logContent := `2024-01-01 10:00:00 [sshd] Ban 192.168.1.1
2024-01-01 10:01:00 [apache] Ban 192.168.1.2
2024-01-01 10:02:00 [sshd] Ban 192.168.1.3
`
t.Run("scan with jail filter", func(t *testing.T) {
scanner := bufio.NewScanner(strings.NewReader(logContent))
config := LogReadConfig{
MaxLines: 10,
JailFilter: "sshd",
}
lines, err := scanLogLines(scanner, config)
assert.NoError(t, err)
assert.Equal(t, 2, len(lines)) // Only sshd lines
for _, line := range lines {
assert.Contains(t, line, "sshd")
}
})
t.Run("scan with IP filter", func(t *testing.T) {
scanner := bufio.NewScanner(strings.NewReader(logContent))
config := LogReadConfig{
MaxLines: 10,
IPFilter: "192.168.1.1",
}
lines, err := scanLogLines(scanner, config)
assert.NoError(t, err)
assert.Len(t, lines, 1)
assert.Contains(t, lines[0], "192.168.1.1")
})
t.Run("scan with both filters", func(t *testing.T) {
scanner := bufio.NewScanner(strings.NewReader(logContent))
config := LogReadConfig{
MaxLines: 10,
JailFilter: "sshd",
IPFilter: "192.168.1.3",
}
lines, err := scanLogLines(scanner, config)
assert.NoError(t, err)
assert.Len(t, lines, 1)
assert.Contains(t, lines[0], "sshd")
assert.Contains(t, lines[0], "192.168.1.3")
})
t.Run("scan with max lines limit", func(t *testing.T) {
scanner := bufio.NewScanner(strings.NewReader(logContent))
config := LogReadConfig{
MaxLines: 1,
}
lines, err := scanLogLines(scanner, config)
assert.NoError(t, err)
assert.Len(t, lines, 1)
})
}
// TestGetCacheStats tests the GetCacheStats function
func TestGetCacheStats(t *testing.T) {
olp := NewOptimizedLogProcessor()
// Initially should have zero stats
hits, misses := olp.GetCacheStats()
assert.Equal(t, int64(0), hits)
assert.Equal(t, int64(0), misses)
}
// TestClearCaches tests the ClearCaches function
func TestClearCaches(t *testing.T) {
olp := NewOptimizedLogProcessor()
// Should not panic
assert.NotPanics(t, func() {
olp.ClearCaches()
})
// Stats should show zero after clear
hits, misses := olp.GetCacheStats()
assert.Equal(t, int64(0), hits)
assert.Equal(t, int64(0), misses)
}
// TestGetLogLinesOptimized tests the GetLogLinesOptimized function
func TestGetLogLinesOptimized(t *testing.T) {
tmpDir := t.TempDir()
oldLogDir := GetLogDir()
SetLogDir(tmpDir)
defer SetLogDir(oldLogDir)
// Create test log file
logFile := filepath.Join(tmpDir, "fail2ban.log")
logContent := `2024-01-01 10:00:00 [sshd] Ban 192.168.1.1
2024-01-01 10:01:00 [apache] Ban 192.168.1.2
`
err := os.WriteFile(logFile, []byte(logContent), 0600)
require.NoError(t, err)
t.Run("successful read with jail filter", func(t *testing.T) {
olp := NewOptimizedLogProcessor()
lines, err := olp.GetLogLinesOptimized("sshd", "", 10)
assert.NoError(t, err)
assert.NotNil(t, lines)
})
t.Run("read with IP filter", func(t *testing.T) {
olp := NewOptimizedLogProcessor()
lines, err := olp.GetLogLinesOptimized("", "192.168.1.1", 10)
assert.NoError(t, err)
assert.NotNil(t, lines)
})
t.Run("read with both filters", func(t *testing.T) {
olp := NewOptimizedLogProcessor()
lines, err := olp.GetLogLinesOptimized("sshd", "192.168.1.1", 5)
assert.NoError(t, err)
assert.NotNil(t, lines)
})
}
// TestGetLogLinesUltraOptimized tests the GetLogLinesUltraOptimized function
func TestGetLogLinesUltraOptimized(t *testing.T) {
tmpDir := t.TempDir()
oldLogDir := GetLogDir()
SetLogDir(tmpDir)
defer SetLogDir(oldLogDir)
// Create test log file
logFile := filepath.Join(tmpDir, "fail2ban.log")
logContent := `2024-01-01 10:00:00 [sshd] Ban 192.168.1.1
2024-01-01 10:01:00 [apache] Ban 192.168.1.2
2024-01-01 10:02:00 [sshd] Ban 192.168.1.3
`
err := os.WriteFile(logFile, []byte(logContent), 0600)
require.NoError(t, err)
t.Run("successful ultra optimized read", func(t *testing.T) {
lines, err := GetLogLinesUltraOptimized("sshd", "", 10)
assert.NoError(t, err)
assert.NotNil(t, lines)
})
t.Run("with both filters", func(t *testing.T) {
lines, err := GetLogLinesUltraOptimized("sshd", "192.168.1.1", 5)
assert.NoError(t, err)
assert.NotNil(t, lines)
})
t.Run("with max lines limit", func(t *testing.T) {
lines, err := GetLogLinesUltraOptimized("", "", 1)
assert.NoError(t, err)
assert.NotNil(t, lines)
})
}
// TestShouldSkipFile tests the shouldSkipFile function
func TestShouldSkipFile(t *testing.T) {
tmpDir := t.TempDir()
// Create test files with different sizes
smallFile := filepath.Join(tmpDir, "small.log")
err := os.WriteFile(smallFile, []byte("small content"), 0600)
require.NoError(t, err)
largeFile := filepath.Join(tmpDir, "large.log")
largeContent := make([]byte, 2*1024*1024) // 2MB
err = os.WriteFile(largeFile, largeContent, 0600)
require.NoError(t, err)
tests := []struct {
name string
filepath string
maxFileSize int64
expectSkip bool
}{
{"small file within limit", smallFile, 1024 * 1024, false},
{"large file exceeds limit", largeFile, 1024 * 1024, true},
{"zero max size - skip nothing", largeFile, 0, false},
{"negative max size - skip nothing", largeFile, -1, false},
{"file exactly at limit", smallFile, 13, false}, // "small content" is 13 bytes
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result := shouldSkipFile(tt.filepath, tt.maxFileSize)
assert.Equal(t, tt.expectSkip, result)
})
}
}
// TestResolveBaseDir tests the resolveBaseDir function
func TestResolveBaseDir(t *testing.T) {
t.Run("from config with absolute path", func(t *testing.T) {
config := LogReadConfig{
BaseDir: "/var/log/fail2ban",
}
result := resolveBaseDir(config)
assert.Equal(t, "/var/log/fail2ban", result)
})
t.Run("from config with empty path uses GetLogDir", func(t *testing.T) {
config := LogReadConfig{
BaseDir: "",
}
result := resolveBaseDir(config)
assert.NotEmpty(t, result)
})
}
// TestStreamLogFileWithContext tests streamLogFileWithContext function
func TestStreamLogFileWithContext(t *testing.T) {
tmpDir := t.TempDir()
logFile := filepath.Join(tmpDir, "test.log")
logContent := `line 1
line 2
line 3
`
err := os.WriteFile(logFile, []byte(logContent), 0600)
require.NoError(t, err)
t.Run("successful stream with context", func(t *testing.T) {
ctx := context.Background()
config := LogReadConfig{
MaxLines: 10,
BaseDir: tmpDir,
}
lines, err := streamLogFileWithContext(ctx, logFile, config)
assert.NoError(t, err)
assert.Len(t, lines, 3)
})
t.Run("context cancellation", func(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
cancel() // Cancel immediately
config := LogReadConfig{
MaxLines: 10,
BaseDir: tmpDir,
}
_, err := streamLogFileWithContext(ctx, logFile, config)
assert.Error(t, err)
assert.Contains(t, err.Error(), "context")
})
t.Run("context timeout", func(t *testing.T) {
ctx, cancel := context.WithTimeout(context.Background(), 1*time.Nanosecond)
defer cancel()
time.Sleep(2 * time.Millisecond) // Ensure timeout
config := LogReadConfig{
MaxLines: 10,
BaseDir: tmpDir,
}
_, err := streamLogFileWithContext(ctx, logFile, config)
assert.Error(t, err)
})
}
// TestCollectLogLines tests the collectLogLines function
func TestCollectLogLines(t *testing.T) {
tmpDir := t.TempDir()
// Create main log file
logFile := filepath.Join(tmpDir, "fail2ban.log")
content := "2024-01-01 10:00:00 [sshd] Ban 192.168.1.1\n"
err := os.WriteFile(logFile, []byte(content), 0600)
require.NoError(t, err)
t.Run("collect from log directory", func(t *testing.T) {
config := LogReadConfig{
MaxLines: 10,
BaseDir: tmpDir,
}
lines, err := collectLogLines(context.Background(), tmpDir, config)
assert.NoError(t, err)
assert.NotNil(t, lines)
})
t.Run("collect with context timeout", func(_ *testing.T) {
ctx, cancel := context.WithTimeout(context.Background(), 1*time.Nanosecond)
defer cancel()
time.Sleep(2 * time.Millisecond)
config := LogReadConfig{
MaxLines: 10,
BaseDir: tmpDir,
}
_, err := collectLogLines(ctx, tmpDir, config)
// May or may not error depending on timing - we're just testing it doesn't panic
_ = err
})
}

63
fail2ban/logs_validation_test.go Normal file
View File

@@ -0,0 +1,63 @@
package fail2ban
import (
"context"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/ivuorinen/f2b/shared"
)
func TestGetLogLinesWithLimit_ValidatesNegativeMaxLines(t *testing.T) {
_, err := GetLogLinesWithLimit(context.Background(), "", "", -1)
require.Error(t, err)
assert.Contains(t, err.Error(), "must be non-negative")
}
func TestGetLogLinesWithLimit_ValidatesExcessiveMaxLines(t *testing.T) {
_, err := GetLogLinesWithLimit(context.Background(), "", "", shared.MaxLogLinesLimit+1)
require.Error(t, err)
assert.Contains(t, err.Error(), "exceeds maximum allowed value")
}
func TestGetLogLinesWithLimit_AcceptsValidMaxLines(t *testing.T) {
// Setup test environment with mock data
cleanup := setupTestLogEnvironment(t, "testdata/fail2ban.log")
defer cleanup()
// Should not error with valid values
_, err := GetLogLinesWithLimit(context.Background(), "", "", 10)
assert.NoError(t, err)
}
func TestGetLogLinesOptimized_ValidatesNegativeMaxLines(t *testing.T) {
olp := &OptimizedLogProcessor{}
_, err := olp.GetLogLinesOptimized("", "", -1)
require.Error(t, err)
assert.Contains(t, err.Error(), "must be non-negative")
}
func TestGetLogLinesOptimized_ValidatesExcessiveMaxLines(t *testing.T) {
olp := &OptimizedLogProcessor{}
_, err := olp.GetLogLinesOptimized("", "", shared.MaxLogLinesLimit+1)
require.Error(t, err)
assert.Contains(t, err.Error(), "exceeds maximum allowed value")
}
func TestGetLogLinesWithLimit_AcceptsZeroMaxLines(t *testing.T) {
// Should return empty slice for zero maxLines
lines, err := GetLogLinesWithLimit(context.Background(), "", "", 0)
assert.NoError(t, err)
assert.Empty(t, lines)
}
func TestGetLogLinesWithLimit_SanitizesFilters(t *testing.T) {
cleanup := setupTestLogEnvironment(t, "testdata/fail2ban.log")
defer cleanup()
// Filters with whitespace should be sanitized
_, err := GetLogLinesWithLimit(context.Background(), " sshd ", " 192.168.1.1 ", 10)
assert.NoError(t, err)
}

10
fail2ban/osrunner_test.go
View File

@@ -43,16 +43,16 @@ func TestGetLogLinesMethod(t *testing.T) {
}
func TestParseUltraOptimized(_ *testing.T) {
// Test ParseBanRecordLineUltraOptimized with simple input
// Test ultra-optimized parsing functions (both singular and plural variants)
line := "192.168.1.1 2025-07-20 12:30:45 2025-07-20 13:30:45"
jail := "sshd"
// Call the function - may fail, that's ok for coverage
_, _ = ParseBanRecordLineUltraOptimized(line, jail)
// Test ParseBanRecordsUltraOptimized (plural)
_, _ = ParseBanRecordsUltraOptimized(line, jail)
// Test with empty line
_, _ = ParseBanRecordLineUltraOptimized("", jail)
_, _ = ParseBanRecordsUltraOptimized("", jail)
// Test with malformed line
// Test ParseBanRecordLineUltraOptimized (singular) with malformed line
_, _ = ParseBanRecordLineUltraOptimized("invalid line", jail)
}

89
fail2ban/security_utils.go Normal file
View File

@@ -0,0 +1,89 @@
// Package fail2ban provides security utility functions for input validation and threat detection.
// This module handles path traversal detection, dangerous command pattern identification,
// and other security-related checks to prevent injection attacks and unauthorized access.
package fail2ban
import (
"path/filepath"
"strings"
)
// ContainsPathTraversal validates paths using stdlib filepath canonicalization.
// Returns true if the path contains traversal attempts (e.g., .., absolute paths, encoded traversals, etc.)
func ContainsPathTraversal(input string) bool {
// Check for URL-encoded or Unicode-encoded traversal attempts
// These are suspicious in path/command contexts and should be rejected
inputLower := strings.ToLower(input)
suspiciousPatterns := []string{
"%2e%2e", // URL encoded ..
"%2f", // URL encoded /
"%5c", // URL encoded \
"\x00", // Null byte
}
for _, pattern := range suspiciousPatterns {
if strings.Contains(inputLower, pattern) {
return true
}
}
// Use filepath.IsLocal (Go 1.20+) to check if path is local and safe
// Returns false for paths that:
// - Are absolute (start with /)
// - Contain .. that escape the current directory
// - Are empty
// - Contain invalid characters
if !filepath.IsLocal(input) {
return true
}
// Additional check: Clean the path and verify it doesn't start with ..
// This catches cases where IsLocal might pass but the path still tries to escape
cleaned := filepath.Clean(input)
if strings.HasPrefix(cleaned, ".."+string(filepath.Separator)) || cleaned == ".." {
return true
}
return false
}
// GetDangerousCommandPatterns returns patterns for log sanitization and threat detection.
//
// Purpose: This list is used for:
// - Sanitizing/masking dangerous patterns in logs to prevent sensitive data leakage
// - Detecting suspicious patterns in command outputs for monitoring/alerting
//
// NOT for: Input validation or injection prevention (use proper validation instead)
//
// The returned patterns include both production patterns (real attack signatures)
// and test sentinels (used exclusively in test fixtures for validation).
func GetDangerousCommandPatterns() []string {
// Production patterns: Real command injection and SQL injection signatures
productionPatterns := []string{
"rm -rf", // Destructive file operations
"drop table", // SQL injection attempts
"'; cat", // Command injection with file reads
"/etc/passwd", "/etc/shadow", // Specific sensitive file access
}
// Test sentinels: Markers used exclusively in test fixtures
// These help verify pattern detection logic in tests
testSentinels := []string{
"DANGEROUS_RM_COMMAND",
"DANGEROUS_SYSTEM_CALL",
"DANGEROUS_COMMAND",
"DANGEROUS_PWD_COMMAND",
"DANGEROUS_LIST_COMMAND",
"DANGEROUS_READ_COMMAND",
"DANGEROUS_OUTPUT_FILE",
"DANGEROUS_INPUT_FILE",
"DANGEROUS_EXEC_COMMAND",
"DANGEROUS_WGET_COMMAND",
"DANGEROUS_CURL_COMMAND",
"DANGEROUS_EXEC_FUNCTION",
"DANGEROUS_SYSTEM_FUNCTION",
"DANGEROUS_EVAL_FUNCTION",
}
// Combine both lists for backward compatibility
return append(productionPatterns, testSentinels...)
}

31
fail2ban/sudo.go
View File

@@ -8,6 +8,8 @@ import (
"os/user"
"sync"
"time"
"github.com/ivuorinen/f2b/shared"
)
const (
@@ -15,18 +17,6 @@ const (
DefaultSudoTimeout = 5 * time.Second
)
// SudoChecker provides methods to check sudo privileges
type SudoChecker interface {
// IsRoot returns true if the current user is root (UID 0)
IsRoot() bool
// InSudoGroup returns true if the current user is in the sudo group
InSudoGroup() bool
// CanUseSudo returns true if the current user can use sudo
CanUseSudo() bool
// HasSudoPrivileges returns true if user has any form of sudo access
HasSudoPrivileges() bool
}
// RealSudoChecker implements SudoChecker using actual system calls
type RealSudoChecker struct{}
@@ -85,7 +75,7 @@ func (r *RealSudoChecker) InSudoGroup() bool {
}
// Check common sudo group names (portable across systems)
if group.Name == "sudo" || group.Name == "wheel" || group.Name == "admin" {
if group.Name == shared.SudoCommand || group.Name == "wheel" || group.Name == "admin" {
return true
}
@@ -108,7 +98,8 @@ func (r *RealSudoChecker) CanUseSudo() bool {
defer cancel()
// Try to run 'sudo -n true' (non-interactive) to test sudo access
cmd := exec.CommandContext(ctx, "sudo", "-n", "true")
// #nosec G204 -- shared.SudoCommand is a hardcoded constant "sudo", not user input
cmd := exec.CommandContext(ctx, shared.SudoCommand, "-n", "true")
err := cmd.Run()
return err == nil
}
@@ -148,14 +139,14 @@ func (m *MockSudoChecker) HasSudoPrivileges() bool {
// RequiresSudo returns true if the given command typically requires sudo privileges
func RequiresSudo(command string, args ...string) bool {
// Commands that typically require sudo for fail2ban operations
if command == Fail2BanClientCommand {
if command == shared.Fail2BanClientCommand {
if len(args) > 0 {
switch args[0] {
case "set", "reload", "restart", "start", "stop":
case shared.ActionSet, shared.ActionReload, shared.ActionRestart, shared.ActionStart, shared.ActionStop:
return true
case "get":
case shared.ActionGet:
// Some get operations might require sudo depending on configuration
if len(args) > 2 && (args[2] == "banip" || args[2] == "unbanip") {
if len(args) > 2 && (args[2] == shared.ActionBanIP || args[2] == shared.ActionUnbanIP) {
return true
}
}
@@ -163,13 +154,13 @@ func RequiresSudo(command string, args ...string) bool {
return false
}
if command == "service" && len(args) > 0 && args[0] == "fail2ban" {
if command == shared.ServiceCommand && len(args) > 0 && args[0] == shared.ServiceFail2ban {
return true
}
if command == "systemctl" && len(args) > 0 {
switch args[0] {
case "start", "stop", "restart", "reload", "enable", "disable":
case shared.ActionStart, "stop", "restart", "reload", "enable", "disable":
return true
}
}

205
fail2ban/sudo_additional_test.go Normal file
View File

@@ -0,0 +1,205 @@
package fail2ban
import (
"testing"
"github.com/stretchr/testify/assert"
)
// TestRealSudoChecker_CanUseSudo_InTestEnvironment tests that CanUseSudo returns false in test environment
func TestRealSudoChecker_CanUseSudo_InTestEnvironment(t *testing.T) {
// Set test environment
t.Setenv("F2B_TEST_SUDO", "1")
checker := &RealSudoChecker{}
result := checker.CanUseSudo()
// Should always return false in test environment (safety measure)
assert.False(t, result, "CanUseSudo should return false in test environment")
}
// TestCanUseSudo_WithMock tests CanUseSudo using mock checker
func TestCanUseSudo_WithMock(t *testing.T) {
tests := []struct {
name string
mockSudo bool
expected bool
}{
{
name: "user can sudo",
mockSudo: true,
expected: true,
},
{
name: "user cannot sudo",
mockSudo: false,
expected: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
mock := &MockSudoChecker{
MockCanUseSudo: tt.mockSudo,
}
result := mock.CanUseSudo()
assert.Equal(t, tt.expected, result,
"MockCanUseSudo=%v should return %v", tt.mockSudo, tt.expected)
})
}
}
// TestMockSudoChecker_CanUseSudo tests the mock implementation
func TestMockSudoChecker_CanUseSudo(t *testing.T) {
mock := &MockSudoChecker{
MockCanUseSudo: true,
}
assert.True(t, mock.CanUseSudo(), "Mock with MockCanUseSudo=true should return true")
mock.MockCanUseSudo = false
assert.False(t, mock.CanUseSudo(), "Mock with MockCanUseSudo=false should return false")
}
// TestHasSudoPrivileges_CanUseSudo tests that CanUseSudo contributes to HasSudoPrivileges
func TestHasSudoPrivileges_CanUseSudo(t *testing.T) {
tests := []struct {
name string
isRoot bool
inSudoGroup bool
canUseSudo bool
expectedPrivilege bool
}{
{
name: "can use sudo only",
isRoot: false,
inSudoGroup: false,
canUseSudo: true,
expectedPrivilege: true,
},
{
name: "cannot use sudo, no other privileges",
isRoot: false,
inSudoGroup: false,
canUseSudo: false,
expectedPrivilege: false,
},
{
name: "can use sudo and is root",
isRoot: true,
inSudoGroup: false,
canUseSudo: true,
expectedPrivilege: true,
},
{
name: "can use sudo and in sudo group",
isRoot: false,
inSudoGroup: true,
canUseSudo: true,
expectedPrivilege: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
mock := &MockSudoChecker{
MockIsRoot: tt.isRoot,
MockInSudoGroup: tt.inSudoGroup,
MockCanUseSudo: tt.canUseSudo,
}
result := mock.HasSudoPrivileges()
assert.Equal(t, tt.expectedPrivilege, result,
"IsRoot=%v, InSudoGroup=%v, CanUseSudo=%v should result in HasSudoPrivileges=%v",
tt.isRoot, tt.inSudoGroup, tt.canUseSudo, tt.expectedPrivilege)
})
}
}
// TestRealSudoChecker_CanUseSudo_Integration tests integration with other sudo checks
func TestRealSudoChecker_CanUseSudo_Integration(t *testing.T) {
// This test ensures CanUseSudo is properly integrated into privilege checking
t.Run("mock checker returns expected values", func(t *testing.T) {
// Create a mock where only CanUseSudo is true
mock := &MockSudoChecker{
MockIsRoot: false,
MockInSudoGroup: false,
MockCanUseSudo: true,
}
// Individual checks should work
assert.False(t, mock.IsRoot())
assert.False(t, mock.InSudoGroup())
assert.True(t, mock.CanUseSudo())
// HasSudoPrivileges should return true (because CanUseSudo is true)
assert.True(t, mock.HasSudoPrivileges(),
"HasSudoPrivileges should be true when CanUseSudo is true")
})
t.Run("explicit privileges override", func(t *testing.T) {
// Test the explicit privileges flag
mock := &MockSudoChecker{
MockIsRoot: false,
MockInSudoGroup: false,
MockCanUseSudo: false,
MockHasPrivileges: true,
ExplicitPrivilegesSet: true,
}
assert.True(t, mock.HasSudoPrivileges(),
"ExplicitPrivilegesSet=true should override computed privileges")
})
}
// TestRealSudoChecker_CanUseSudo_TestEnvironmentDetection tests test environment detection
func TestRealSudoChecker_CanUseSudo_TestEnvironmentDetection(t *testing.T) {
tests := []struct {
name string
envVar string
envValue string
shouldBlock bool
}{
{
name: "F2B_TEST_SUDO set",
envVar: "F2B_TEST_SUDO",
envValue: "1",
shouldBlock: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Setenv(tt.envVar, tt.envValue)
checker := &RealSudoChecker{}
result := checker.CanUseSudo()
assert.False(t, result, "Should return false in test environment")
})
}
}
// TestCanUseSudo_MockConsistency tests that mock behavior is consistent
func TestCanUseSudo_MockConsistency(t *testing.T) {
// Test that setting/unsetting the mock produces expected results
originalChecker := GetSudoChecker()
defer SetSudoChecker(originalChecker)
t.Run("mock set to true", func(t *testing.T) {
mock := &MockSudoChecker{MockCanUseSudo: true}
SetSudoChecker(mock)
checker := GetSudoChecker()
assert.True(t, checker.CanUseSudo(), "Should return true when mock is set to true")
})
t.Run("mock set to false", func(t *testing.T) {
mock := &MockSudoChecker{MockCanUseSudo: false}
SetSudoChecker(mock)
checker := GetSudoChecker()
assert.False(t, checker.CanUseSudo(), "Should return false when mock is set to false")
})
}

124
fail2ban/test_helpers.go
View File

@@ -6,6 +6,8 @@ import (
"path/filepath"
"strings"
"testing"
"github.com/ivuorinen/f2b/shared"
)
// TestingInterface represents the common interface between testing.T and testing.B
@@ -23,14 +25,14 @@ func setupTestLogEnvironment(t *testing.T, testDataFile string) (cleanup func())
// Validate test data file exists and is safe to read
absTestLogFile, err := filepath.Abs(testDataFile)
if err != nil {
t.Fatalf("Failed to get absolute path: %v", err)
t.Fatalf(shared.ErrFailedToGetAbsPath, err)
}
if _, err := os.Stat(absTestLogFile); os.IsNotExist(err) {
t.Skipf("Test data file not found: %s", absTestLogFile)
t.Skipf(shared.ErrTestDataNotFound, absTestLogFile)
}
// Ensure the file is within testdata directory for security
if !strings.Contains(absTestLogFile, "testdata") {
if !strings.Contains(absTestLogFile, shared.TestDataDir) {
t.Fatalf("Test file must be in testdata directory: %s", absTestLogFile)
}
@@ -43,7 +45,7 @@ func setupTestLogEnvironment(t *testing.T, testDataFile string) (cleanup func())
if err != nil {
t.Fatalf("Failed to read test file: %v", err)
}
if err := os.WriteFile(mainLog, data, 0600); err != nil {
if err := os.WriteFile(mainLog, data, shared.DefaultFilePermissions); err != nil {
t.Fatalf("Failed to create test log: %v", err)
}
@@ -76,21 +78,18 @@ func SetupMockEnvironment(t TestingInterface) (client *MockClient, cleanup func(
SetRunner(mockRunner)
// Configure comprehensive mock responses
mockRunner.SetResponse("fail2ban-client -V", []byte("fail2ban-client v0.11.2"))
mockRunner.SetResponse(
"fail2ban-client status",
[]byte("Status\n|- Number of jail:\t2\n`- Jail list:\tsshd, apache"),
)
mockRunner.SetResponse("fail2ban-client ping", []byte("pong"))
mockRunner.SetResponse(shared.MockCommandVersion, []byte(shared.VersionOutput))
mockRunner.SetResponse(shared.MockCommandStatus, []byte(shared.StatusOutput))
mockRunner.SetResponse(shared.MockCommandPing, []byte(shared.PingOutput))
// Standard jail responses
mockRunner.SetResponse("fail2ban-client status sshd", []byte("Status for the jail: sshd"))
mockRunner.SetResponse("fail2ban-client status apache", []byte("Status for the jail: apache"))
mockRunner.SetResponse(shared.MockCommandStatusSSHD, []byte("Status for the jail: sshd"))
mockRunner.SetResponse(shared.MockCommandStatusApache, []byte("Status for the jail: apache"))
// Standard ban responses
mockRunner.SetResponse("fail2ban-client set sshd banip 192.168.1.100", []byte("0"))
mockRunner.SetResponse("fail2ban-client set sshd unbanip 192.168.1.100", []byte("0"))
mockRunner.SetResponse("fail2ban-client banned 192.168.1.100", []byte("[]"))
mockRunner.SetResponse(shared.MockCommandBanIP, []byte(shared.Fail2BanStatusSuccess))
mockRunner.SetResponse(shared.MockCommandUnbanIP, []byte(shared.Fail2BanStatusSuccess))
mockRunner.SetResponse(shared.MockCommandBanned, []byte(shared.MockBannedOutput))
cleanup = func() {
SetSudoChecker(originalChecker)
@@ -121,12 +120,9 @@ func SetupMockEnvironmentWithSudo(t TestingInterface, hasSudo bool) (client *Moc
// Configure mock responses based on sudo availability
if hasSudo {
mockRunner.SetResponse("fail2ban-client -V", []byte("fail2ban-client v0.11.2"))
mockRunner.SetResponse("fail2ban-client ping", []byte("pong"))
mockRunner.SetResponse(
"fail2ban-client status",
[]byte("Status\n|- Number of jail:\t2\n`- Jail list:\tsshd, apache"),
)
mockRunner.SetResponse(shared.MockCommandVersion, []byte(shared.VersionOutput))
mockRunner.SetResponse(shared.MockCommandPing, []byte(shared.PingOutput))
mockRunner.SetResponse(shared.MockCommandStatus, []byte(shared.StatusOutput))
}
cleanup = func() {
@@ -151,10 +147,10 @@ func SetupBasicMockClient() *MockClient {
func AssertError(t TestingInterface, err error, expectError bool, testName string) {
t.Helper()
if expectError && err == nil {
t.Fatalf("%s: expected error but got none", testName)
t.Fatalf(shared.ErrTestExpectedError, testName)
}
if !expectError && err != nil {
t.Fatalf("%s: unexpected error: %v", testName, err)
t.Fatalf(shared.ErrTestUnexpected, testName, err)
}
}
@@ -173,10 +169,10 @@ func AssertErrorContains(t TestingInterface, err error, expectedSubstring string
func AssertCommandSuccess(t TestingInterface, err error, output, expectedOutput, testName string) {
t.Helper()
if err != nil {
t.Fatalf("%s: unexpected error: %v, output: %s", testName, err, output)
t.Fatalf(shared.ErrTestUnexpectedWithOutput, testName, err, output)
}
if expectedOutput != "" && !strings.Contains(output, expectedOutput) {
t.Fatalf("%s: expected output to contain %q, got: %s", testName, expectedOutput, output)
t.Fatalf(shared.ErrTestExpectedOutput, testName, expectedOutput, output)
}
}
@@ -194,7 +190,7 @@ func AssertCommandError(t TestingInterface, err error, output, expectedError, te
// createTestGzipFile creates a gzip file with given content for testing
func createTestGzipFile(t TestingInterface, path string, content []byte) {
// Validate path is safe for test file creation
if !strings.Contains(path, os.TempDir()) && !strings.Contains(path, "testdata") {
if !strings.Contains(path, os.TempDir()) && !strings.Contains(path, shared.TestDataDir) {
t.Fatalf("Test file path must be in temp directory or testdata: %s", path)
}
@@ -226,7 +222,7 @@ func setupTempDirWithFiles(t TestingInterface, files map[string][]byte) string {
for filename, content := range files {
path := filepath.Join(tempDir, filename)
if err := os.WriteFile(path, content, 0600); err != nil {
if err := os.WriteFile(path, content, shared.DefaultFilePermissions); err != nil {
t.Fatalf("Failed to create file %s: %v", filename, err)
}
}
@@ -239,10 +235,10 @@ func validateTestDataFile(t *testing.T, testDataFile string) string {
t.Helper()
absTestLogFile, err := filepath.Abs(testDataFile)
if err != nil {
t.Fatalf("Failed to get absolute path: %v", err)
t.Fatalf(shared.ErrFailedToGetAbsPath, err)
}
if _, err := os.Stat(absTestLogFile); os.IsNotExist(err) {
t.Skipf("Test data file not found: %s", absTestLogFile)
t.Skipf(shared.ErrTestDataNotFound, absTestLogFile)
}
return absTestLogFile
}
@@ -265,3 +261,73 @@ func assertContainsText(t *testing.T, lines []string, text string) {
}
t.Errorf("Expected to find '%s' in results", text)
}
// StandardMockSetup configures comprehensive standard responses for MockRunner
// This eliminates the need for repetitive SetResponse calls in individual tests
func StandardMockSetup(mockRunner *MockRunner) {
// Version responses
mockRunner.SetResponse("fail2ban-client -V", []byte(shared.MockVersion))
mockRunner.SetResponse("sudo fail2ban-client -V", []byte(shared.MockVersion))
// Ping responses
mockRunner.SetResponse("fail2ban-client ping", []byte(shared.PingOutput))
mockRunner.SetResponse("sudo fail2ban-client ping", []byte(shared.PingOutput))
// Status responses
statusResponse := "Status\n|- Number of jail: 2\n`- Jail list: sshd, apache"
mockRunner.SetResponse("fail2ban-client status", []byte(statusResponse))
mockRunner.SetResponse("sudo fail2ban-client status", []byte(statusResponse))
// Individual jail status responses
sshdStatus := "Status for the jail: sshd\n|- Filter\n| |- Currently failed:\t0\n| " +
"|- Total failed:\t5\n| `- File list:\t/var/log/auth.log\n`- Actions\n " +
"|- Currently banned:\t1\n |- Total banned:\t2\n `- Banned IP list:\t192.168.1.100"
mockRunner.SetResponse(shared.MockCommandStatusSSHD, []byte(sshdStatus))
mockRunner.SetResponse("sudo "+shared.MockCommandStatusSSHD, []byte(sshdStatus))
apacheStatus := "Status for the jail: apache\n|- Filter\n| |- Currently failed:\t0\n| " +
"|- Total failed:\t3\n| `- File list:\t/var/log/apache2/error.log\n`- Actions\n " +
"|- Currently banned:\t0\n |- Total banned:\t1\n `- Banned IP list:\t"
mockRunner.SetResponse(shared.MockCommandStatusApache, []byte(apacheStatus))
mockRunner.SetResponse("sudo "+shared.MockCommandStatusApache, []byte(apacheStatus))
// Ban/unban responses
mockRunner.SetResponse(shared.MockCommandBanIP, []byte(shared.Fail2BanStatusSuccess))
mockRunner.SetResponse("sudo "+shared.MockCommandBanIP, []byte(shared.Fail2BanStatusSuccess))
mockRunner.SetResponse(shared.MockCommandUnbanIP, []byte(shared.Fail2BanStatusSuccess))
mockRunner.SetResponse("sudo "+shared.MockCommandUnbanIP, []byte(shared.Fail2BanStatusSuccess))
mockRunner.SetResponse("fail2ban-client set apache banip 192.168.1.101", []byte(shared.Fail2BanStatusSuccess))
mockRunner.SetResponse("sudo fail2ban-client set apache banip 192.168.1.101", []byte(shared.Fail2BanStatusSuccess))
mockRunner.SetResponse("fail2ban-client set apache unbanip 192.168.1.101", []byte(shared.Fail2BanStatusSuccess))
mockRunner.SetResponse(
"sudo fail2ban-client set apache unbanip 192.168.1.101",
[]byte(shared.Fail2BanStatusSuccess),
)
// Banned IP responses
mockRunner.SetResponse("fail2ban-client banned 192.168.1.100", []byte(shared.MockBannedOutput))
mockRunner.SetResponse("sudo fail2ban-client banned 192.168.1.100", []byte(shared.MockBannedOutput))
mockRunner.SetResponse("fail2ban-client banned 192.168.1.101", []byte("[]"))
mockRunner.SetResponse("sudo fail2ban-client banned 192.168.1.101", []byte("[]"))
}
// SetupMockEnvironmentWithStandardResponses combines mock environment setup with standard responses
// This is a convenience function for tests that need comprehensive mock responses
func SetupMockEnvironmentWithStandardResponses(t TestingInterface) (client *MockClient, cleanup func()) {
t.Helper()
client, cleanup = SetupMockEnvironment(t)
// Safe type assertion with error handling
mockRunner, ok := GetRunner().(*MockRunner)
if !ok {
t.Fatalf("Expected GetRunner() to return *MockRunner, got %T", GetRunner())
}
StandardMockSetup(mockRunner)
return client, cleanup
}

36
fail2ban/time_parser.go
View File

@@ -1,35 +1,44 @@
package fail2ban
import (
"fmt"
"strings"
"sync"
"time"
"github.com/ivuorinen/f2b/shared"
)
// TimeParsingCache provides cached and optimized time parsing functionality
// TimeParsingCache provides cached and optimized time parsing functionality with bounded cache
type TimeParsingCache struct {
layout string
parseCache sync.Map // string -> time.Time
parseCache *BoundedTimeCache // Bounded cache prevents unbounded memory growth
stringBuilder sync.Pool
}
// NewTimeParsingCache creates a new time parsing cache with the specified layout
func NewTimeParsingCache(layout string) *TimeParsingCache {
func NewTimeParsingCache(layout string) (*TimeParsingCache, error) {
parseCache, err := NewBoundedTimeCache(shared.CacheMaxSize)
if err != nil {
return nil, fmt.Errorf("failed to create time parsing cache: %w", err)
}
return &TimeParsingCache{
layout: layout,
layout: layout,
parseCache: parseCache, // Bounded at 10k entries
stringBuilder: sync.Pool{
New: func() interface{} {
return &strings.Builder{}
},
},
}
}, nil
}
// ParseTime parses a time string with caching for performance
// ParseTime parses a time string with bounded caching for performance
func (tpc *TimeParsingCache) ParseTime(timeStr string) (time.Time, error) {
// Check cache first
if cached, ok := tpc.parseCache.Load(timeStr); ok {
return cached.(time.Time), nil
return cached, nil
}
// Parse and cache
@@ -54,10 +63,19 @@ func (tpc *TimeParsingCache) BuildTimeString(dateStr, timeStr string) string {
// Global cache instances for common time formats
var (
defaultTimeCache = NewTimeParsingCache("2006-01-02 15:04:05")
defaultTimeCache = mustCreateTimeCache()
)
// ParseBanTime parses ban time using the default cache
// mustCreateTimeCache creates the default time cache or panics (init time only)
func mustCreateTimeCache() *TimeParsingCache {
cache, err := NewTimeParsingCache("2006-01-02 15:04:05")
if err != nil {
panic(fmt.Sprintf("failed to create default time cache: %v", err))
}
return cache
}
// ParseBanTime parses ban time using the default bounded cache
func ParseBanTime(timeStr string) (time.Time, error) {
return defaultTimeCache.ParseTime(timeStr)
}

57
fail2ban/types.go Normal file
View File

@@ -0,0 +1,57 @@
// Package fail2ban defines common data structures and types.
// This package provides core types used throughout the fail2ban integration,
// including ban records, configuration structures, and logging interfaces.
package fail2ban
import (
"time"
)
// BanRecord represents a single ban entry with jail, IP, ban time, and remaining duration.
type BanRecord struct {
Jail string
IP string
BannedAt time.Time
Remaining string
}
// Fields represents a map of structured log fields (decoupled from logrus)
type Fields map[string]interface{}
// LoggerEntry represents a structured logging entry that can be chained
type LoggerEntry interface {
WithField(key string, value interface{}) LoggerEntry
WithFields(fields Fields) LoggerEntry
WithError(err error) LoggerEntry
Debug(args ...interface{})
Info(args ...interface{})
Warn(args ...interface{})
Error(args ...interface{})
Debugf(format string, args ...interface{})
Infof(format string, args ...interface{})
Warnf(format string, args ...interface{})
Errorf(format string, args ...interface{})
}
// LoggerInterface defines the top-level logging interface (decoupled from logrus)
type LoggerInterface interface {
WithField(key string, value interface{}) LoggerEntry
WithFields(fields Fields) LoggerEntry
WithError(err error) LoggerEntry
Debug(args ...interface{})
Info(args ...interface{})
Warn(args ...interface{})
Error(args ...interface{})
Debugf(format string, args ...interface{})
Infof(format string, args ...interface{})
Warnf(format string, args ...interface{})
Errorf(format string, args ...interface{})
}
// LogCollectionConfig configures log line collection behavior
type LogCollectionConfig struct {
Jail string
IP string
MaxLines int
MaxFileSize int64
}

202
fail2ban/validation_cache.go Normal file
View File

@@ -0,0 +1,202 @@
// Package fail2ban provides validation caching utilities for performance optimization.
// This module handles caching of validation results to avoid repeated expensive validation
// operations, with metrics support and thread-safe cache management.
package fail2ban
import (
"context"
"sync"
"github.com/ivuorinen/f2b/shared"
)
// ValidationCache provides thread-safe caching for validation results with bounded size.
// The cache automatically evicts entries when it reaches capacity to prevent memory exhaustion.
type ValidationCache struct {
mu sync.RWMutex
cache map[string]error
}
// NewValidationCache creates a new bounded validation cache.
// The cache will automatically evict entries when it reaches capacity to prevent
// unbounded memory growth in long-running processes. See constants.go for cache limits.
func NewValidationCache() *ValidationCache {
return &ValidationCache{
cache: make(map[string]error),
}
}
// Get retrieves a cached validation result
func (vc *ValidationCache) Get(key string) (bool, error) {
vc.mu.RLock()
defer vc.mu.RUnlock()
result, exists := vc.cache[key]
return exists, result
}
// Set stores a validation result in the cache.
// If the cache is at capacity, it automatically evicts a portion of entries.
// Invalid keys (empty or too long) are silently ignored to prevent cache pollution.
func (vc *ValidationCache) Set(key string, err error) {
// Validate key before locking to prevent cache pollution
if key == "" || len(key) > 512 {
return // Invalid key - skip caching
}
vc.mu.Lock()
defer vc.mu.Unlock()
// Evict if at or above max to ensure bounded size
if len(vc.cache) >= shared.CacheMaxSize {
vc.evictEntries()
}
vc.cache[key] = err
}
// evictEntries removes a portion of cache entries to free up space.
// Must be called with vc.mu held (Lock, not RLock).
// Evicts entries based on shared.CacheEvictionRate using random iteration.
func (vc *ValidationCache) evictEntries() {
targetSize := int(float64(len(vc.cache)) * (1.0 - shared.CacheEvictionRate))
count := 0
// Go map iteration is random, so this effectively evicts random entries
for key := range vc.cache {
if len(vc.cache) <= targetSize {
break
}
delete(vc.cache, key)
count++
}
// Log eviction for observability (optional, could use metrics)
if count > 0 {
getLogger().WithField("evicted", count).WithField("remaining", len(vc.cache)).
Debug("Validation cache evicted entries")
}
}
// Clear removes all entries from the cache
func (vc *ValidationCache) Clear() {
vc.mu.Lock()
defer vc.mu.Unlock()
// Create a new map instead of deleting entries for better performance
vc.cache = make(map[string]error)
}
// Size returns the number of entries in the cache
func (vc *ValidationCache) Size() int {
vc.mu.RLock()
defer vc.mu.RUnlock()
return len(vc.cache)
}
// Global validation caches for frequently used validators
var (
ipValidationCache = NewValidationCache()
jailValidationCache = NewValidationCache()
filterValidationCache = NewValidationCache()
commandValidationCache = NewValidationCache()
// metricsRecorder is set by the cmd package to avoid circular dependencies
metricsRecorder MetricsRecorder
metricsRecorderMu sync.RWMutex
)
// SetMetricsRecorder sets the metrics recorder (called by cmd package)
func SetMetricsRecorder(recorder MetricsRecorder) {
metricsRecorderMu.Lock()
defer metricsRecorderMu.Unlock()
metricsRecorder = recorder
}
// getMetricsRecorder returns the current metrics recorder
func getMetricsRecorder() MetricsRecorder {
metricsRecorderMu.RLock()
defer metricsRecorderMu.RUnlock()
return metricsRecorder
}
// cachedValidate provides a generic caching wrapper for validation functions.
// Context parameter supports cancellation and timeout for validation operations.
func cachedValidate(
ctx context.Context,
cache *ValidationCache,
keyPrefix string,
value string,
validator func(string) error,
) error {
// Check context cancellation before expensive operations
if ctx.Err() != nil {
return ctx.Err()
}
cacheKey := keyPrefix + ":" + value
if exists, result := cache.Get(cacheKey); exists {
// Record cache hit in metrics
if recorder := getMetricsRecorder(); recorder != nil {
recorder.RecordValidationCacheHit()
}
return result
}
// Record cache miss in metrics
if recorder := getMetricsRecorder(); recorder != nil {
recorder.RecordValidationCacheMiss()
}
// Check context again before calling validator
if ctx.Err() != nil {
return ctx.Err()
}
err := validator(value)
cache.Set(cacheKey, err)
return err
}
// CachedValidateIP validates an IP address with caching.
// Context parameter supports cancellation and timeout for validation operations.
func CachedValidateIP(ctx context.Context, ip string) error {
return cachedValidate(ctx, ipValidationCache, "ip", ip, ValidateIP)
}
// CachedValidateJail validates a jail name with caching.
// Context parameter supports cancellation and timeout for validation operations.
func CachedValidateJail(ctx context.Context, jail string) error {
return cachedValidate(ctx, jailValidationCache, string(shared.ContextKeyJail), jail, ValidateJail)
}
// CachedValidateFilter validates a filter name with caching.
// Context parameter supports cancellation and timeout for validation operations.
func CachedValidateFilter(ctx context.Context, filter string) error {
return cachedValidate(ctx, filterValidationCache, "filter", filter, ValidateFilter)
}
// CachedValidateCommand validates a command with caching.
// Context parameter supports cancellation and timeout for validation operations.
func CachedValidateCommand(ctx context.Context, command string) error {
return cachedValidate(ctx, commandValidationCache, string(shared.ContextKeyCommand), command, ValidateCommand)
}
// ClearValidationCaches clears all validation caches
func ClearValidationCaches() {
ipValidationCache.Clear()
jailValidationCache.Clear()
filterValidationCache.Clear()
commandValidationCache.Clear()
}
// GetValidationCacheStats returns statistics for all validation caches
func GetValidationCacheStats() map[string]int {
return map[string]int{
"ip_cache_size": ipValidationCache.Size(),
"jail_cache_size": jailValidationCache.Size(),
"filter_cache_size": filterValidationCache.Size(),
"command_cache_size": commandValidationCache.Size(),
}
}

43
fail2ban/validation_cache_test.go
View File

@@ -1,6 +1,8 @@
package fail2ban
import (
"context"
"fmt"
"sync"
"testing"
)
@@ -40,7 +42,7 @@ func TestValidationCaching(t *testing.T) {
tests := []struct {
name string
validator func(string) error
validator func(context.Context, string) error
validInput string
expectedHits int
expectedMisses int
@@ -87,13 +89,13 @@ func TestValidationCaching(t *testing.T) {
ClearValidationCaches()
// First call - should be a cache miss
err := tt.validator(tt.validInput)
err := tt.validator(context.Background(), tt.validInput)
if err != nil {
t.Fatalf("First validation call failed: %v", err)
}
// Second call - should be a cache hit
err = tt.validator(tt.validInput)
err = tt.validator(context.Background(), tt.validInput)
if err != nil {
t.Fatalf("Second validation call failed: %v", err)
}
@@ -128,7 +130,7 @@ func TestValidationCacheConcurrency(t *testing.T) {
defer wg.Done()
for j := 0; j < numCallsPerGoroutine; j++ {
// Use the same IP to test caching
err := CachedValidateIP("192.168.1.1")
err := CachedValidateIP(context.Background(), "192.168.1.1")
if err != nil {
t.Errorf("Concurrent validation failed: %v", err)
return
@@ -172,13 +174,13 @@ func TestValidationCacheInvalidInput(t *testing.T) {
invalidIP := "invalid.ip.address"
// First call - should be a cache miss and return error
err1 := CachedValidateIP(invalidIP)
err1 := CachedValidateIP(context.Background(), invalidIP)
if err1 == nil {
t.Fatal("Expected error for invalid IP, got none")
}
// Second call - should be a cache hit and return the same error
err2 := CachedValidateIP(invalidIP)
err2 := CachedValidateIP(context.Background(), invalidIP)
if err2 == nil {
t.Fatal("Expected error for invalid IP on second call, got none")
}
@@ -206,13 +208,13 @@ func BenchmarkValidationCaching(b *testing.B) {
validIP := "192.168.1.1"
// Warm up the cache
_ = CachedValidateIP(validIP)
_ = CachedValidateIP(context.Background(), validIP)
b.ResetTimer()
b.RunParallel(func(pb *testing.PB) {
for pb.Next() {
// All calls should hit the cache
_ = CachedValidateIP(validIP)
_ = CachedValidateIP(context.Background(), validIP)
}
})
}
@@ -227,3 +229,28 @@ func BenchmarkValidationNoCaching(b *testing.B) {
}
})
}
// TestValidationCacheEviction tests that cache eviction works correctly
func TestValidationCacheEviction(t *testing.T) {
cache := NewValidationCache()
// Fill cache to trigger eviction (using CacheMaxSize from shared package)
// Add significantly more than maxSize to guarantee eviction
entriesToAdd := 11000 // CacheMaxSize is 10000
for i := 0; i < entriesToAdd; i++ {
// Add unique keys to cache
key := fmt.Sprintf("test-key-%d", i)
cache.Set(key, nil) // nil means valid
}
// Verify cache was evicted and didn't grow unbounded
sizeAfter := cache.Size()
if sizeAfter > 10000 {
t.Errorf("Cache should have evicted entries to stay under 10000, got: %d", sizeAfter)
}
if sizeAfter == 0 {
t.Errorf("Cache should not be empty after eviction, got size: %d", sizeAfter)
}
t.Logf("Cache evicted successfully after adding %d entries: final size %d", entriesToAdd, sizeAfter)
}