Updated code to correctly format and write products to JSON file.
Performance Improvements:
- Moved regex compilation to package level (only compiled once)
- Added a custom HTTP client with proper timeouts and connection pooling
- Used buffer pool to reduce memory allocations
- Used
io.Copy
instead ofioutil.ReadAll
for better memory efficiency
Reliability Improvements:
- Added context with timeout
- Added status code checking
- Implemented exponential backoff retry mechanism
- Better error handling with wrapped errors
Code Quality Improvements:
- More descriptive error messages
- Structured logging with additional context
- Better resource cleanup with defers
- Consistent error message formatting