Skip to content

Latest commit

 

History

History
1278 lines (1061 loc) · 39.5 KB

PRD.md

File metadata and controls

1278 lines (1061 loc) · 39.5 KB

byDefault Project PRD

Overview

byDefault is a Python-based CLI tool for Splunk 9.2.2 Technology Add-on (TA) development and maintenance, focusing on configuration file management and standardization.

Developer Workflow

The following example demonstrates how byDefault integrates into a typical development workflow:

# Create and switch to new feature branch
$ git checkout -b feat/new-dashboard

# Start Splunk and make changes in the GUI
$ splunk start
... make changes in Splunk Web UI ...
$ splunk stop

# Check what changes were made
$ bydefault scan
Changes detected in: my_custom_ta
  Modified files:
    local/props.conf
      [new_sourcetype] - New stanza
      [existing_sourcetype] - Modified
    local/transforms.conf
      [lookup_transform] - New stanza
    local/data/lookups/new_lookup.csv - New file

# Merge changes to default directory
$ bydefault merge
Merging changes in: my_custom_ta
  ✓ props.conf: 2 stanzas merged
  ✓ transforms.conf: 1 stanza merged
  ✓ data/lookups/new_lookup.csv copied

# Review sorted configurations
$ bydefault sort default/props.conf
Sorting: default/props.conf
  ✓ Stanzas reordered: 5
  ✓ Settings sorted: 23

# Bump version across all TAs
$ bydefault bumpver --minor
Updating versions:
  my_custom_ta: 1.2.0 -> 1.3.0
  another_ta: 2.1.0 -> 2.2.0

# Commit changes and create PR
$ git add .
$ git commit -m "feat: add new sourcetype and lookup

- Added new sourcetype configuration
- Created lookup transform
- Added lookup table
- Bumped TA versions"
$ git push origin feat/new-dashboard

Requirements

Core Functionality

  1. Phased Implementation Approach:

    Scope Note: The project is divided into four distinct phases, each building upon the previous. Upon completion of each phase, the tool should be functional with the core requirements of the phase. Later phases should be modular and extend the functionality of previous phases. The phases are as follows:

    • P1: Basic CLI Structure

      • Setup command-line interface framework
      • Define all commands and their options
      • Implement help text and documentation
      • Create output formatting templates
      • Setup error message formatting
      • Implement verbosity controls

    • P2: Basic Validation

      • Implement core validation rules
      • Setup validation command structure
      • Create validation reporting
      • Enable automatic validation in other commands
      • Define validation error handling

    • P3: File Detection

      • Basic file I/O operations
      • File type detection and validation
      • TA directory validation
      • Change detection and reporting
      • Local/default file comparison
      • Status display system

    • P4: Configuration Sorting

      • Stanza sorting by type and priority
      • Setting organization within stanzas
      • Structure and comment preservation
      • Format maintenance

    • P5: Configuration Merging

      • Local to default file merging
      • Conflict detection and resolution
      • Format and structure preservation
      • Backup mechanisms

  2. Command Structure:

    # Root command help
$ bydefault --help
Usage: bydefault [OPTIONS] COMMAND [ARGS]...

CLI tools for Splunk TA development and maintenance.

Options:
  --version  Show version information
  --help     Show this message and exit

Commands:
  scan      Detect and report configuration changes
  sort      Sort configuration files maintaining structure
  merge     Merge local configurations into default
  validate  Verify configuration structure and syntax
  bumpver   Update version numbers across TAs

# Command-specific help examples
$ bydefault scan --help
Usage: bydefault scan [OPTIONS]

Detect and report configuration changes.

Options:
  --verbose       Show detailed output
  --dry-run      Show what would be done without making changes
  --recursive    Scan subdirectories for TAs
  --show-diff    Show detailed changes
  --include-meta Include metadata file changes
  --help         Show this message and exit

$ bydefault sort --help
Usage: bydefault sort [OPTIONS] FILE

Sort configuration files maintaining structure.

Options:
  --verbose    Show detailed output
  --dry-run    Show what would be done without making changes
  --backup     Create backup before sorting
  --verify     Verify file structure after sort
  --help       Show this message and exit

$ bydefault merge --help
Usage: bydefault merge [OPTIONS] TA_PATH

Merge local configurations into default directory.

Options:
  --verbose    Show detailed output
  --dry-run    Show what would be done without making changes
  --backup     Create backup before merging
  --conflict   Specify conflict resolution strategy
  --help       Show this message and exit

$ bydefault validate --help
Usage: bydefault validate [OPTIONS] [FILES]...

Verify configuration structure and syntax.

Options:
  --verbose    Show detailed output
  --dry-run    Show what would be done without making changes
  --strict     Enable strict validation rules
  --report     Generate validation report
  --help       Show this message and exit

$ bydefault bumpver --help
Usage: bydefault bumpver [OPTIONS] [TA_PATHS]...

Update version numbers across TAs.

Options:
  --verbose    Show detailed output
  --dry-run    Show what would be done without making changes
  --major      Increment major version (X.0.0)
  --minor      Increment minor version (0.X.0)
  --patch      Increment patch version (0.0.X)
  --set VERSION Set specific version (X.Y.Z)
  --help       Show this message and exit

Technical Requirements

  1. Project Structure:

    bydefault/
├── src/
│   └── bydefault/
│       ├── __init__.py
│       ├── cli.py              # Main CLI entry point and group definitions
│       ├── commands/           # Command implementations
│       │   ├── __init__.py
│       │   ├── scan.py
│       │   ├── sort.py
│       │   ├── merge.py
│       │   ├── validate.py
│       │   └── version.py
│       ├── core/              # Core business logic
│       │   ├── __init__.py
│       │   ├── scanner.py     # File detection logic
│       │   ├── sorter.py      # Sorting implementation
│       │   ├── merger.py      # Merge logic
│       │   └── validator.py   # Validation rules
│       ├── models/            # Data models
│       │   ├── __init__.py
│       │   ├── stanza.py
│       │   └── changes.py
│       └── utils/             # Shared utilities
│           ├── __init__.py
│           ├── config.py      # Configuration handling
│           ├── files.py       # File operations
│           └── output.py      # Terminal output formatting
├── tests/
│   ├── conftest.py
│   ├── test_commands/        # Command-specific tests
│   ├── test_core/           # Core logic tests
│   ├── test_models/         # Model tests
│   └── test_utils/          # Utility tests
└── [core config files]

  2. Component Responsibilities:

    a. Commands (commands/):

    • Handles CLI argument parsing and validation
    • Manages user interaction and feedback
    • Formats and displays output
    • Provides error handling for users
    • Thin wrapper around core functionality

    b. Core Logic (core/):

    • Implements business logic in isolation
    • Processes data without user interaction
    • Returns structured data (no printing/output)
    • Can be used programmatically
    • Handles complex operations

    c. Data Models (models/):

    • Defines data structures used throughout application
    • Provides type hints and validation
    • Ensures consistent data representation
    • Handles data transformations
    • Documents data relationships

    d. Utilities (utils/):

    • Implements shared functionality
    • Provides common operations
    • Handles cross-cutting concerns
    • Maintains consistent interfaces
    • Reusable across components

  3. Dependencies:

    • Python >= 3.11

    • Standard Library:

      • configparser >= 5.3.0 (INI file handling)
        • Documentation
        • Native .conf file parsing
        • Preserves file structure
        • Handles multi-line values
        • Standard library integration

    • External Core Libraries:

      • click >= 8.1.0 (CLI framework)

        • Documentation
        • Provides robust CLI interface
        • Includes file handling utilities
        • Built-in help text generation
        • Type validation and conversion

      • rich-click >= 1.8.3 (CLI formatting)

        • Documentation
        • Enhanced Click help formatting
        • Consistent styling across commands
        • Built-in Rich integration
        • Drop-in Click wrapper

      • rich >= 13.9.4 (Terminal formatting)

        • Documentation
        • Clear status output
        • Structured error messages
        • Progress indicators
        • Consistent formatting

    • External Development and Distribution:

      • uv >= 0.5.1 (Package and environment management)

        • Documentation
        • Virtual environment management
        • Package installation and management
        • Tool distribution
        • Dependency resolution

      • ruff >= 0.7.3 (Linting and formatting)

        • Documentation
        • Code formatting
        • Import sorting
        • Code linting
        • Style enforcement

    • External Testing Tools:

      • pytest >= 8.3.3 (Testing framework)

        • Documentation
        • Test discovery and execution
        • Fixture management
        • Assertion introspection

      • pytest-cov >= 6.0.0 (Coverage reporting)

        • Documentation
        • Coverage measurement
        • Report generation
        • Coverage enforcement

      • pytest-mock >= 3.14.0 (Mocking in tests)

        • Documentation
        • Function/method mocking
        • Call verification
        • Return value simulation

  4. Interface Requirements:

    • CLI Framework (Click):
      • Automatic help text generation
      • Command grouping and nesting
      • Type validation and conversion
      • Error handling and reporting
    • Global Options:
      • --verbose: Enable detailed output
      • --dry-run: Show changes without applying
      • --version: Show version information
    • Output Formatting:
      • Consistent status indicators
      • Color-coded output (when supported)
      • Progress indicators for long operations
      • Clear error messages

  5. Performance Targets:

    • Startup time < 100ms
    • Scan operation < 1s for single TA
    • Memory usage < 100MB for typical operations
    • Responsive CLI feedback for all operations

Implementation Rules

Example Project Structure:

main_splunk_application/             # Root project directory
├── TA-example-main/                 # Primary TA with full structure
│   ├── README.md
│   ├── default/
│   │   ├── app.conf                 # Required TA configuration
│   │   ├── props.conf               # Source type definitions
│   │   ├── transforms.conf          # Data transformations
│   │   ├── eventtypes.conf          # Event type definitions
│   │   ├── tags.conf                # Tag configurations
│   │   └── data/
│   │       └── lookups/             # Default lookup files
│   │           ├── example.csv
│   │           └── mappings.csv
│   ├── local/                       # Local overwrites
│   │   ├── props.conf
│   │   └── transforms.conf
│   ├── metadata/
│   │   ├── local.meta               # Local metadata
│   │   └── default.meta             # Default metadata
│   ├── bin/
│   │   └── scripts/                 # TA-specific scripts
│   │       ├── __init__.py
│   │       └── example.py
│   └── static/                      # Static assets
│       └── appIcon.png
│
├── TA-example-secondary/            # Additional TA (minimal structure)
│
└── TA-example-dev/                  # Development TA (minimal structure)
│
└── TA-example-other-TA/             # Other TA (minimal structure)
│
└── SA-example-with-diff-name/       # Source-type TA with different name
│
└── SA-example-with-diff-name-2/     # Source-type TA with different name
│
└── DA-example-diff-name/            # Destination-type TA with different name

  1. Phase-Aligned Processing:

    a. P1 - Basic CLI Structure:

    1. Command Framework:

      • Define root command structure
      • Setup global options (--verbose, --dry-run, --version)
      • Implement subcommand scaffolding
      • Create help text templates

    2. Output Formatting:

      • Define status message templates
      • Setup color schemes for different message types
      • Create progress indicator formats
      • Implement verbosity output levels:
        • Default: Summary information only
        • Verbose: Detailed operation information

    Example Commands:

     $ bydefault --help
 Usage: bydefault [OPTIONS] COMMAND [ARGS]...

 CLI tools for Splunk TA development and maintenance.

 Options:
   --verbose    Enable detailed output
   --dry-run    Show what would be done without making changes
   --version    Show version information
   --help       Show this message and exit

 Commands:
   scan        Detect and report configuration changes
   sort        Sort configuration files maintaining structure
   merge       Merge local configurations into default
   validate    Verify configuration structure and syntax
   bumpver     Update version numbers across TAs

    b. P2 - Basic Validation:

    1. Implement core validation rules
    2. Setup validation command structure
    3. Create validation reporting
    4. Enable automatic validation in other commands
    5. Define validation error handling

    c. P3 - File Detection:

    1. TA Structure Validation:

      • Verify app.conf presence and basic structure
      • Validate required directory presence (local/default)
      • Check for metadata directory
      • Report structural issues

    2. Change Detection:

      • Scan for local/ directories and local.meta files
      • Check .gitignore for local directory exceptions
      • Report changes at TA level:
        • Default: Show files and stanzas changed
        • Verbose: Option to display file contents
      • Track detected changes:
        • File presence in local/
        • Stanza presence in local files
        • Basic change summary

    Example Usage:

     $ bydefault scan
 Scanning for changes...
 Changes detected in: my_custom_ta
   Modified files:
     local/props.conf
       [new_sourcetype] - New stanza
       [existing_sourcetype] - Modified
     local/transforms.conf
       [lookup_transform] - New stanza

 $ bydefault scan --verbose my_custom_ta
 Scanning my_custom_ta for changes...
 Found local/props.conf:
   [new_sourcetype]
     TRANSFORMS-example = new_example
     SHOULD_LINEMERGE = false
   [existing_sourcetype]
     # Modified from: TRANSFORMS-old = old_value
     TRANSFORMS-old = new_value
 Found local/transforms.conf:
   [lookup_transform]
     filename = new_lookup.csv
     max_matches = 1
 ```

d. P4 - Configuration Sorting:
1. Stanza Classification and Order:
   - Primary Order (Highest to Lowest Priority):
     - Global settings (no stanza header)
     - Default stanza: Exact match "[default]"
     - Global wildcard stanzas: Match pattern "[*::attribute]"
     - Specific type stanzas: All "[type::*]" and "[type::attribute]" patterns

   - Secondary Order (within specific type groups):
     - Wildcard attribute "[type::*]"
     - Wildcard-prefixed attribute "[type::*-attribute]"
     - Specific attribute "[type::attribute]"

   ```# TODO: Add specific sorting rules for settings within stanzas```

2. Comment Preservation Rules:
   - Comments directly above stanza: Associated with stanza
   - Comments after stanza header: Prompt for association
   - Comments directly above key=value: Associated with that pair
   - Inline value comments: Move with the value
   - Comments followed by blank line and another stanza: Associated with previous stanza block

   Example Comment Associations:

   ```ini
   # Comment for stanza1
   [stanza1]
   # This comment could be for the stanza or for setting1 - prompt user
   setting1 = value1  # inline comment
   setting2 = value2

   # This comment belongs to stanza1 because it's followed by a blank line
   # and another stanza
   
   [stanza2]
   setting1 = value1
   ```

   ```# TODO: Add more complex comment association examples```

3. Processing Scope:
   - Support both single file and full TA sorting
   - Apply to configuration and metadata files only
   - Maintain consistent sorting across all TAs
   - Run automatically after merge operations

Example Usage:

```bash
 $ bydefault sort default/props.conf
 Sorting: default/props.conf
   ✓ Global settings reordered
   ✓ [default] stanza positioned
   ✓ 3 wildcard stanzas sorted
   ✓ 12 specific stanzas sorted
   ✓ Comments preserved

$ bydefault sort --verbose default/props.conf
Sorting: default/props.conf
  Processing global settings...
    ✓ 3 settings reordered
  Positioning [default] stanza...
    ✓ Stanza moved to line 12
  Sorting wildcard stanzas...
    ✓ [*::example] moved to line 25
    ✓ [*::test] moved to line 32
    ✓ [*::production] moved to line 39
  Sorting specific stanzas...
    ✓ [source::*] group sorted
    ✓ [host::*] group sorted
    ✓ [sourcetype::*] group sorted
  Preserving comments...
    ✓ 15 comment blocks maintained

    e. P5 - Configuration Merging:

    1. Merge Scope:

      • Support merging at multiple levels:
        • Full project (all TAs)
        • Single TA (by path or name)
        • Single configuration file
      • Process configuration and metadata files:
        • Configuration files:
          • app.conf # Required TA configuration
          • inputs.conf # Data input configurations
          • props.conf # Source type definitions
          • transforms.conf # Data transformations
          • outputs.conf # Data forwarding settings
          • eventtypes.conf # Event type definitions
          • macros.conf # Search macros
          • tags.conf # Tag definitions
          • savedsearches.conf # Saved searches
        • Metadata files:
          • local.meta
          • default.meta

      Note: For a complete list of configuration files, see: https://docs.splunk.com/Documentation/Splunk/9.2.2/Admin/Wheretofindtheconfigurationfiles

      • Remove local files/directories after successful merge
      • Support --dry-run for verification

    2. Merge Process:

      • For each stanza in local:
        • Update existing key/value pairs in default
        • Add new key/value pairs to default
        • Retain existing default values not in local
      • Maintain stanza structure and comments
      • Remove local files after successful merge

      Example Merge Operation:

      # default/props.conf (before)
[mystanza]
TRANSFORMS-example = example
SHOULD_LINEMERGE = false

# local/props.conf
[mystanza]
TRANSFORMS-example = new_example
EVAL-new_field = value

# default/props.conf (after)
[mystanza]
TRANSFORMS-example = new_example
SHOULD_LINEMERGE = false
EVAL-new_field = value

    3. Operation Output:

      • Display simple success/failure summary
      • Show count of merged files and stanzas
      • Indicate removal of local files
      • Provide detailed output with --verbose

      Example Output:

      $ bydefault merge my_custom_ta
Merging changes in: my_custom_ta
  ✓ props.conf: 2 stanzas merged
  ✓ transforms.conf: 1 stanza merged
Local files removed

Safety, Validation, and Error Handling

  1. Backup Strategy:

    • Create backups only for file-altering operations (e.g., sort, merge) unless --dry-run is used.
    • Maintain only the latest backup per operation.
    • Store backups in a dedicated directory with clear naming conventions.

    Example Backup Operation:

    $ bydefault merge my_custom_ta
Creating backup: .bydefault/backups/merge_2024-03-22_142015/
  ✓ default/props.conf -> props.conf.bak
  ✓ default/transforms.conf -> transforms.conf.bak
Proceeding with merge...

  2. Validation Rules:

    • Apply basic validation for all configuration and metadata files:
      • Ensure valid stanza format and proper key-value syntax.
      • Use static rulesets for .conf and .meta files.
    • Automatically validate during file-altering commands.
    • Allow ad-hoc validation using the validate command.

    Example Validation:

    $ bydefault validate default/props.conf
Validating: default/props.conf
  ✓ File encoding: UTF-8
  ✓ Stanza format: Valid
  ✓ Key-value pairs: Valid
  ✓ Structure: Valid

$ bydefault validate default/invalid.conf
Validating: default/invalid.conf
  ✗ Error on line 15: Invalid stanza format
  ✗ Error on line 23: Missing value for key 'TRANSFORMS'
Validation failed: 2 errors found

  3. Error Reporting:

    • Provide clear and concise error messages for end-users.
    • Include file path and line number for validation errors.
    • Differentiate error types (e.g., command not found, file not found, validation errors).

    Example Error Messages:

    $ bydefault merge nonexistent_ta
Error: TA not found - 'nonexistent_ta' does not exist or is not a valid TA

$ bydefault sort default/props.conf
Error in default/props.conf (line 45):
  Invalid stanza format: Missing closing bracket ']'
Operation aborted: File validation failed

$ bydefault merge my_custom_ta
Error: Permission denied
  Cannot write to default/props.conf
  Please check file permissions and try again

  4. Logging and Auditing:

    • Implement basic logging for file-altering operations.
    • Logs should be accessible to end-users.
    • Expand logging capabilities as the tool matures.

  5. Recovery Process:

    • Enable automatic restoration from backups in case of failure.
    • Provide detailed failure logs for troubleshooting.
    • Offer user guidance for manual recovery steps if needed.

  6. Operation Standards:

    • Ensure atomic writes to prevent partial updates.
    • Maintain operation history for audit purposes.
    • Log validation results and any error conditions.

Testing Requirements

  1. Phase-Specific Testing:

    a. P1 - Basic CLI Testing:

    • Command Registration:
      • Verify all commands are registered
      • Test help text for each command
      • Validate option handling
    • Global Options:
      • Test --verbose output levels
      • Verify --dry-run behavior
      • Check version display
    • Output Formatting:
      • Verify color schemes
      • Test progress indicators
      • Check error message formatting

    b. P2 - Basic Validation Testing:

    • Validation Rules:
      • Test stanza format validation
      • Verify key-value pair syntax
      • Check file encoding detection
    • Validation Command:
      • Test standalone validation
      • Verify validation reporting
      • Check error handling
    • Edge Cases:
      • Test empty files
      • Verify handling of malformed content
      • Check unicode handling

    c. P3 - File Detection Testing:

    • TA Structure:
      • Verify app.conf detection
      • Test directory structure validation
      • Check metadata handling
    • Change Detection:
      • Test local file identification
      • Verify stanza change detection
      • Check .gitignore integration
    • Status Display:
      • Test default output format
      • Verify verbose output
      • Check error conditions
    • Validation Integration:
      • Verify validation runs during detection
      • Test invalid file handling
      • Check validation error reporting

    d. P4 - Configuration Sorting Testing:

    • Stanza Ordering:
      • Test primary sort order
      • Verify secondary sort rules
      • Check stanza grouping
    • Comment Handling:
      • Test comment association rules
      • Verify comment preservation
      • Check multi-line comments
    • File Processing:
      • Test single file sorting
      • Verify full TA sorting
      • Check metadata file sorting
    • Validation Integration:
      • Test validation before sorting
      • Verify sort aborts on validation failure
      • Check validation error handling during sort

    e. P5 - Configuration Merging Testing:

    • Merge Operations:
      • Test file-level merging
      • Verify stanza-level merging
      • Check setting-level merging
    • Backup System:
      • Test backup creation
      • Verify backup restoration
      • Check cleanup processes
    • Error Recovery:
      • Test partial merge recovery
      • Verify rollback functionality
      • Check error reporting
    • Validation Integration:
      • Test validation before merge
      • Verify merge aborts on validation failure
      • Check validation error handling during merge

  2. Cross-Phase Testing:

    • Command Integration:
      • Test command sequences
      • Verify state preservation
      • Check error propagation
    • Data Integrity:
      • Verify file content preservation
      • Test configuration validity
      • Check metadata consistency
    • Performance:
      • Test resource usage
      • Verify operation timeouts
      • Check memory constraints

  3. Test Environment Requirements:

    • Directory Structures:
      • Single TA setup
      • Multiple TA project
      • Various configuration types
    • File Variations:
      • Different configuration files
      • Various metadata structures
      • Edge case examples
    • Error Conditions:
      • Permission issues
      • Invalid configurations
      • Incomplete structures

Documentation Requirements

  1. Code Documentation:

    • Python Docstrings:

      • Use comprehensive docstrings for all functions and classes
      • Include type hints for all parameters and returns
      • Document exceptions that may be raised
      • Provide usage examples for complex functionality
      • Follow PEP 257 docstring conventions

      Example Docstring:

      def validate_stanza(content: str, file_category: str = "conf") -> ValidationResult:
    """Validate basic stanza structure and syntax.

    Args:
        content: The text content to validate as a stanza
        file_category: The type of file being validated ("conf" or "meta")

    Returns:
        ValidationResult: Object containing validation status and any errors

    Raises:
        ValidationError: If stanza structure is invalid
        ValueError: If file_category is not "conf" or "meta"

    Example:
        >>> result = validate_stanza("[stanza_name]\\nkey = value")
        >>> assert result.is_valid
        >>> result = validate_stanza("[invalid", file_category="meta")
        >>> assert not result.is_valid
        >>> assert "Missing closing bracket" in result.errors
    """

  2. User Documentation:

    • Installation and Setup:

      • Environment requirements
      • UV installation steps
      • Initial configuration
      • Verification steps

    • Command Reference:

      • Detailed description of each command
      • All available options and flags
      • Example usage with output
      • Common use cases

    • Workflow Examples:

      • Basic TA development workflow
      • Configuration management scenarios
      • Error recovery procedures
      • Best practices

  3. Development Documentation:

    • Architecture Overview:

      • Component relationships
      • Data flow diagrams
      • Key abstractions
      • Extension points

    • Contributing Guidelines:

      • Development setup
      • Testing procedures
      • Code style requirements
      • PR process

  4. Maintenance Documentation:

    • Version History:

      • Detailed changelog
      • Migration guides
      • Breaking changes
      • Deprecation notices

    • Troubleshooting Guide:

      • Common issues
      • Error message explanations
      • Recovery procedures
      • Support resources

Error Handling Requirements

  1. Error Categories and Examples:

    a. CLIErrors:

    • Invalid command usage
    • Unknown options
    • Missing required arguments

    Example:

    $ bydefault sort
Error: Missing argument 'FILE'
  sort command requires a configuration file path
  Try 'bydefault sort --help' for usage information

    b. ValidationErrors:

    • Invalid stanza format
    • Malformed key-value pairs
    • Unsupported file type

    Example:

    $ bydefault validate default/props.conf
Error in default/props.conf (line 23):
  Invalid stanza format: Missing closing bracket ']'
  [sourcetype:my_source
                     ^

    c. FileSystemErrors:

    • File not found
    • Permission denied
    • Invalid file structure

    Example:

    $ bydefault merge my_ta
Error: Invalid TA structure
  Directory 'my_ta' is missing required 'default' folder
  Expected structure:
    my_ta/
    ├── default/
    │   └── app.conf
    └── local/

    d. OperationErrors:

    • Merge conflicts
    • Sort failures
    • Backup failures

    Example:

    $ bydefault merge my_ta
Error: Backup creation failed
  Could not create backup directory: Permission denied
  Please check permissions for .bydefault/backups/
Operation aborted: No changes made

  2. Error Recovery Procedures:

    • Automatic rollback for failed operations
    • Backup restoration for file alterations
    • Clear user guidance for resolution
    • Operation state logging

  3. Error Reporting Standards:

    • Include error location (file, line)
    • Provide context for the error
    • Suggest resolution steps
    • Show command help when relevant

  4. Recovery Validation:

    • Verify system state after recovery
    • Confirm backup restoration
    • Check file integrity
    • Report recovery status

Performance Requirements

  1. Core Requirements:

    • Modularity:

      • Keep components loosely coupled
      • Enable easy feature additions
      • Support future extensibility
      • Allow component replacement

    • Scalability:

      • Handle multiple TAs efficiently
      • Process large configuration files
      • Support batch operations
      • Maintain performance with increased load

    • Operation Timeouts:

      • Default timeout: 30 seconds per operation
      • Configurable via environment variable
      • Clear timeout messages
      • Clean process termination

    Example Timeout:

    $ bydefault merge large_ta
Error: Operation timed out after 30 seconds
  Consider breaking operation into smaller chunks
  Or adjust BYDEFAULT_TIMEOUT environment variable

Future Considerations

  1. Development Tools:

    • Scaffold command for new TA creation
    • Template system for common configurations
    • Project-wide search and replace functionality

  2. Validation Extensions:

    • Serverclass configuration validation
    • Cross-TA dependency checking
    • Custom validation rule definitions

  3. Maintenance Tools:

    • Cleanup command for stale configurations
    • Usage analysis for TAs and stanzas
    • Deployment status tracking

  4. Integration Features:

    • Remote operations support
    • CI/CD pipeline integration
    • Splunk Cloud compatibility

  5. Performance Enhancements:

    • Parallel processing for large projects
    • Incremental backup system
    • Configuration state caching:
      • Cache validated file structures
      • Store sorted stanza orders
      • Remember recent change detections

    Example Cache Usage:

    $ bydefault sort default/props.conf
Using cached validation results...
Sorting: default/props.conf
  ✓ Stanzas reordered: 5
  ✓ Settings sorted: 23

$ bydefault scan
Using cached TA structure...
Changes detected since last scan:
  Modified files:
    local/props.conf

  6. Future Platform Support:

    • Windows compatibility
    • Symlink handling

Success Criteria

  1. Phase Completion:

    • P1: CLI Framework

      • All commands properly registered and documented
      • Consistent output formatting across commands
      • Proper handling of global options
      • Clear error message presentation

    • P2: Basic Validation

      • Reliable stanza structure validation
      • Extensible validation API
      • Clear validation error reporting
      • Proper handling of edge cases

    • P3: File Detection

      • Accurate TA structure identification
      • Reliable change detection
      • Integration with validation command
      • Clear change reporting
      • Proper handling of local/default comparison

    • P4: Configuration Sorting

      • Correct stanza ordering
      • Integration with validation command
      • Reliable comment preservation
      • Consistent sorting across file types
      • Proper handling of complex configurations

    • P5: Configuration Merging

      • Accurate merging of configurations
      • Integration with validation command
      • Clean local file removal
      • Reliable backup creation
      • Proper error recovery

  2. Core Requirements:

    • Reliability:

      • No data loss during operations
      • Failures should be safe
        • Changes are only made after all validation checks pass
        • Failed operations must not leave partial results
        • Any started operations must complete or be fully reverted on failure
      • Consistent behavior across environments
      • Proper error handling and recovery
      • Accurate configuration processing

    • Usability:

      • Clear command structure
      • Helpful error messages
      • Intuitive workflow
      • Comprehensive help text

Development Phases

P1: Basic CLI (Initial Release)

Core Implementation:

  • Command-line interface framework
    • Global options setup (--version, --help)
    • Command-specific options setup (--verbose, --dry-run)
    • Help text system
    • Output formatting templates
    • Error message system
  • Project structure setup
    • Directory organization
    • Package configuration
    • Development tooling setup

Success Criteria:

  1. CLI Framework

    • Root command returns proper help text
    • Global flags work as expected (--version, --help)
    • Command-specific flags work as expected (--verbose, --dry-run)
    • Version information is displayed correctly
    • All help text clearly indicates which commands are not yet implemented

  2. Output System

    • Consistent formatting across all messages
    • Error messages follow defined template
    • Color schemes work in supported terminals

  3. Testing

    • CLI initialization has 100% test coverage
    • Help text verification tests
    • Global flag behavior tests
    • Output formatting tests

P2: Basic Validation

Core Implementation:

  • Implement core validation rules
  • Setup validation command structure
  • Create validation reporting
  • Enable automatic validation in other commands
  • Define validation error handling

Success Criteria:

  1. Validation Rules
    • Implement core validation rules
    • Setup validation command structure
    • Create validation reporting
    • Enable automatic validation in other commands
    • Define validation error handling

P3: File Detection

Core Implementation:

  • Basic file I/O operations
    • Can identify Splunk configuration files
    • Can read file contents without parsing
    • Supports recursive directory scanning
    • Handles file access errors gracefully
    • File reading and validation
    • Error handling for file operations
    • File type detection
  • TA directory validation
  • Change detection and reporting
  • Local/default file comparison
  • Status display system

Success Criteria:

  1. File Operations

    • Reliable file reading and validation
    • Proper error handling for file operations
    • File type detection accuracy
    • Performance within specified limits

  2. Change Detection

    • Identifies new files in local directory
    • Detects modified stanzas
    • Reports untracked files
    • Shows metadata changes when requested

  3. Status Output

    • Clear, git-style status display
    • Grouped by TA when multiple present
    • Shows file-level changes
    • Shows stanza-level changes

  4. Testing

    • Tests for all change scenarios
    • Directory structure validation tests
    • Status output format tests
    • Error condition tests

P4: Configuration Sorting

Core Implementation:

  • Stanza sorting by type and priority
  • Setting organization within stanzas
  • Structure and comment preservation
  • Format maintenance

Success Criteria:

  1. Stanza Sorting

    • Follows defined sort order:
      1. Global settings (no stanza)
      2. [default] stanza
      3. Global wildcards [*::attribute]
      4. Type-specific stanzas
    • Maintains stanza grouping
    • Preserves comments and spacing

  2. Setting Organization

    • Consistent key-value pair ordering
    • Preserves multi-line values
    • Maintains inline comments
    • Keeps original formatting

  3. Testing

    • Sort order verification tests
    • Format preservation tests
    • Comment handling tests
    • Edge case handling tests

P5: Configuration Merging

Core Implementation:

  • Local to default file merging
  • Conflict detection and resolution
  • Format and structure preservation
  • Backup system

Success Criteria:

  1. Merge Operations

    • Successfully merges local changes to default
    • Handles conflicting changes
    • Preserves file structure
    • Creates reliable backups

  2. Conflict Resolution

    • Identifies merge conflicts
    • Provides resolution options
    • Maintains data integrity
    • Logs all merge decisions

  3. Testing

    • Merge scenario tests
    • Conflict resolution tests
    • Backup system tests
    • Recovery scenario tests