Files
gibidify/fileproc/cache.go
Ismo Vuorinen 95b7ef6dd3 chore: modernize workflows, security scanning, and linting configuration (#50)
* build: update Go 1.25, CI workflows, and build tooling

- Upgrade to Go 1.25
- Add benchmark targets to Makefile
- Implement parallel gosec execution
- Lock tool versions for reproducibility
- Add shellcheck directives to scripts
- Update CI workflows with improved caching

* refactor: migrate from golangci-lint to revive

- Replace golangci-lint with revive for linting
- Configure comprehensive revive rules
- Fix all EditorConfig violations
- Add yamllint and yamlfmt support
- Remove deprecated .golangci.yml

* refactor: rename utils to shared and deduplicate code

- Rename utils package to shared
- Add shared constants package
- Deduplicate constants across packages
- Address CodeRabbit review feedback

* fix: resolve SonarQube issues and add safety guards

- Fix all 73 SonarQube OPEN issues
- Add nil guards for resourceMonitor, backpressure, metricsCollector
- Implement io.Closer for headerFileReader
- Propagate errors from processing helpers
- Add metrics and templates packages
- Improve error handling across codebase

* test: improve test infrastructure and coverage

- Add benchmarks for cli, fileproc, metrics
- Improve test coverage for cli, fileproc, config
- Refactor tests with helper functions
- Add shared test constants
- Fix test function naming conventions
- Reduce cognitive complexity in benchmark tests

* docs: update documentation and configuration examples

- Update CLAUDE.md with current project state
- Refresh README with new features
- Add usage and configuration examples
- Add SonarQube project configuration
- Consolidate config.example.yaml

* fix: resolve shellcheck warnings in scripts

- Use ./*.go instead of *.go to prevent dash-prefixed filenames
  from being interpreted as options (SC2035)
- Remove unreachable return statement after exit (SC2317)
- Remove obsolete gibidiutils/ directory reference

* chore(deps): upgrade go dependencies

* chore(lint): megalinter fixes

* fix: improve test coverage and fix file descriptor leaks

- Add defer r.Close() to fix pipe file descriptor leaks in benchmark tests
- Refactor TestProcessorConfigureFileTypes with helper functions and assertions
- Refactor TestProcessorLogFinalStats with output capture and keyword verification
- Use shared constants instead of literal strings (TestFilePNG, FormatMarkdown, etc.)
- Reduce cognitive complexity by extracting helper functions

* fix: align test comments with function names

Remove underscores from test comments to match actual function names:
- benchmark/benchmark_test.go (2 fixes)
- fileproc/filetypes_config_test.go (4 fixes)
- fileproc/filetypes_registry_test.go (6 fixes)
- fileproc/processor_test.go (6 fixes)
- fileproc/resource_monitor_types_test.go (4 fixes)
- fileproc/writer_test.go (3 fixes)

* fix: various test improvements and bug fixes

- Remove duplicate maxCacheSize check in filetypes_registry_test.go
- Shorten long comment in processor_test.go to stay under 120 chars
- Remove flaky time.Sleep in collector_test.go, use >= 0 assertion
- Close pipe reader in benchmark_test.go to fix file descriptor leak
- Use ContinueOnError in flags_test.go to match ResetFlags behavior
- Add nil check for p.ui in processor_workers.go before UpdateProgress
- Fix resource_monitor_validation_test.go by setting hardMemoryLimitBytes directly

* chore(yaml): add missing document start markers

Add --- document start to YAML files to satisfy yamllint:
- .github/workflows/codeql.yml
- .github/workflows/build-test-publish.yml
- .github/workflows/security.yml
- .github/actions/setup/action.yml

* fix: guard nil resourceMonitor and fix test deadlock

- Guard resourceMonitor before CreateFileProcessingContext call
- Add ui.UpdateProgress on emergency stop and path error returns
- Fix potential deadlock in TestProcessFile using wg.Go with defer close
2025-12-10 19:07:11 +02:00

131 lines
3.1 KiB
Go

// Package fileproc handles file processing, collection, and output formatting.
package fileproc
// getNormalizedExtension efficiently extracts and normalizes the file extension with caching.
func (r *FileTypeRegistry) getNormalizedExtension(filename string) string {
// Try cache first (read lock)
r.cacheMutex.RLock()
if ext, exists := r.extCache[filename]; exists {
r.cacheMutex.RUnlock()
return ext
}
r.cacheMutex.RUnlock()
// Compute normalized extension
ext := normalizeExtension(filename)
// Cache the result (write lock)
r.cacheMutex.Lock()
// Check cache size and clean if needed
if len(r.extCache) >= r.maxCacheSize*2 {
r.clearExtCache()
r.stats.CacheEvictions++
}
r.extCache[filename] = ext
r.cacheMutex.Unlock()
return ext
}
// getFileTypeResult gets cached file type detection result or computes it.
func (r *FileTypeRegistry) getFileTypeResult(filename string) FileTypeResult {
ext := r.getNormalizedExtension(filename)
// Update statistics
r.updateStats(func() {
r.stats.TotalLookups++
})
// Try cache first (read lock)
r.cacheMutex.RLock()
if result, exists := r.resultCache[ext]; exists {
r.cacheMutex.RUnlock()
r.updateStats(func() {
r.stats.CacheHits++
})
return result
}
r.cacheMutex.RUnlock()
// Cache miss
r.updateStats(func() {
r.stats.CacheMisses++
})
// Compute result
result := FileTypeResult{
Extension: ext,
IsImage: r.imageExts[ext],
IsBinary: r.binaryExts[ext],
Language: r.languageMap[ext],
}
// Handle special cases for binary detection (like .DS_Store)
if !result.IsBinary && isSpecialFile(filename, r.binaryExts) {
result.IsBinary = true
}
// Cache the result (write lock)
r.cacheMutex.Lock()
if len(r.resultCache) >= r.maxCacheSize {
r.clearResultCache()
r.stats.CacheEvictions++
}
r.resultCache[ext] = result
r.cacheMutex.Unlock()
return result
}
// clearExtCache clears half of the extension cache (LRU-like behavior).
func (r *FileTypeRegistry) clearExtCache() {
r.clearCache(&r.extCache, r.maxCacheSize)
}
// clearResultCache clears half of the result cache.
func (r *FileTypeRegistry) clearResultCache() {
newCache := make(map[string]FileTypeResult, r.maxCacheSize)
count := 0
for k, v := range r.resultCache {
if count >= r.maxCacheSize/2 {
break
}
newCache[k] = v
count++
}
r.resultCache = newCache
}
// clearCache is a generic cache clearing function.
func (r *FileTypeRegistry) clearCache(cache *map[string]string, maxSize int) {
newCache := make(map[string]string, maxSize)
count := 0
for k, v := range *cache {
if count >= maxSize/2 {
break
}
newCache[k] = v
count++
}
*cache = newCache
}
// invalidateCache clears both caches when the registry is modified.
func (r *FileTypeRegistry) invalidateCache() {
r.cacheMutex.Lock()
defer r.cacheMutex.Unlock()
r.extCache = make(map[string]string, r.maxCacheSize)
r.resultCache = make(map[string]FileTypeResult, r.maxCacheSize)
r.stats.CacheEvictions++
}
// updateStats safely updates statistics.
func (r *FileTypeRegistry) updateStats(fn func()) {
r.cacheMutex.Lock()
fn()
r.cacheMutex.Unlock()
}