Skip to content

Latest commit

 

History

History
181 lines (127 loc) · 5.78 KB

testing.md

File metadata and controls

181 lines (127 loc) · 5.78 KB

BlockchainDB Testing Documentation

This document provides an overview of the testing approach used in BlockchainDB, evaluating what works well and what could be improved.

Overview of Testing Approach

BlockchainDB uses Go's standard testing framework along with the testify package for assertions. The tests are primarily focused on functional verification and performance benchmarking of the various components.

Test Files Structure

The repository contains the following test files:

  • bfile_test.go - Tests for the Buffered File component
  • bloom_test.go - Tests for the Bloom Filter component
  • fastrandom_test.go - Tests for the random number generator used in testing
  • history_file_test.go - Tests for the History File component
  • keys_test.go - Tests for key operations
  • kfile_header_test.go - Tests for the KFile header
  • kfile_test.go - Tests for the Key File component
  • kv_2_test.go - Tests for the KV2 implementation
  • kv_shard_test.go - Tests for the KV Shard implementation
  • kv_test.go - Tests for the main Key-Value store
  • view_kv_test.go - Tests for the View KV implementation

Testing Utilities

The codebase includes several testing utilities:

  1. MakeDir - Creates a temporary directory for testing and returns a function to clean it up
  2. MakeFilename - Creates a temporary file for testing
  3. FastRandom - A deterministic random number generator for creating test data

What Works Well

1. Comprehensive Component Testing

Each major component of BlockchainDB has dedicated test files that verify its functionality. This ensures that individual components work as expected before they're integrated.

2. Performance Measurement

Many tests include performance measurements, reporting metrics like operations per second. This helps track performance characteristics and identify regressions.

Example from kv_test.go:

wps := cntWrites / time.Since(start).Seconds()
rps := cntReads / time.Since(start).Seconds()
fmt.Printf("Writes per second %10.3f Reads per second %10.3f\n", wps, rps)

3. Data Integrity Verification

Tests verify data integrity by writing values and then reading them back to ensure they match. This is crucial for a database system.

Example from kv_test.go:

value2, err := kv.Get(key)
assert.NoError(t, err, "Failed to put")
assert.Equal(t, value, value2, "Didn't the the value back")

4. Edge Case Testing

Some tests check edge cases such as:

  • Reading after closing and reopening files
  • Handling non-existent keys
  • Testing compression operations
  • Verifying behavior with large datasets

5. Cleanup Mechanisms

Tests use defer statements with cleanup functions to ensure test resources are properly released, even if tests fail.

Example:

dir, rm := MakeDir()
defer rm()

Areas for Improvement

1. Test Coverage Reporting

The repository doesn't appear to include test coverage reporting. Adding coverage analysis would help identify untested code paths.

Recommendation:

go test -coverprofile=coverage.out ./...
go tool cover -html=coverage.out

2. Mocking External Dependencies

The tests directly interact with the filesystem, which can make tests slower and less reliable. Introducing interfaces and mocks for external dependencies would improve test isolation.

3. Table-Driven Tests

Many tests could benefit from a table-driven approach, which would make it easier to test multiple scenarios with less code duplication.

Example improvement:

func TestBloom(t *testing.T) {
    testCases := []struct {
        name          string
        numEntries    int
        falsePositive float64
    }{
        {"small", 1000, 0.1},
        {"medium", 10000, 0.05},
        {"large", 100000, 0.01},
    }
    
    for _, tc := range testCases {
        t.Run(tc.name, func(t *testing.T) {
            bloom := NewBloom(4.5)
            fr := NewFastRandom([]byte{1})
            
            // Test implementation...
        })
    }
}

4. Benchmarking

While some tests measure performance, formal Go benchmarks using the testing.B type are not extensively used. Adding dedicated benchmark functions would provide more consistent performance measurements.

Example:

func BenchmarkKVPut(b *testing.B) {
    dir, rm := MakeDir()
    defer rm()
    
    kv, _ := NewKV(false, dir, 1024, 10000, 50)
    fr := NewFastRandom([]byte{1})
    
    b.ResetTimer()
    for i := 0; i < b.N; i++ {
        key := fr.NextHash()
        value := fr.RandBuff(100, 200)
        kv.Put(key, value)
    }
}

5. Integration Testing

Most tests focus on individual components. Adding more integration tests that verify the interaction between components would improve confidence in the system as a whole.

6. Parameterized Tests

Some tests use hard-coded values for parameters like buffer sizes. Parameterizing these tests would help identify optimal configurations and edge cases.

7. Error Injection

Adding tests that deliberately inject errors (like disk full scenarios or corrupted files) would help verify error handling paths.

Test Execution

To run the tests in BlockchainDB:

# Run all tests
go test ./...

# Run tests for a specific package
go test github.com/AccumulateNetwork/BlockchainDB/database

# Run a specific test
go test -run TestKV github.com/AccumulateNetwork/BlockchainDB/database

# Run tests with verbose output
go test -v ./...

# Run tests and generate coverage report
go test -coverprofile=coverage.out ./...
go tool cover -html=coverage.out -o coverage.html

Conclusion

The BlockchainDB testing approach provides good functional verification of components and includes performance measurements. However, there are opportunities to improve test coverage, isolation, and consistency through more structured testing approaches and better use of Go's testing features.