Merge pull request #4 from Codium-ai/add-tests-docs

Add comprehensive documentation for test generation feature in Codiumate

docs/tests/behaviors.md

@@ -0,0 +1,27 @@
+# Understanding Behaviors in Test Generation
+
+## Overview
+In the journey of test generation, Codiumate employs a sophisticated approach to analyze your code thoroughly. It extends beyond mere syntax analysis, incorporating an understanding of dependencies and imports to grasp the full context of your code's functionality. This comprehensive analysis leads to the identification of various behaviors your code exhibits.
+
+## Behavior Categories
+Codiumate categorizes identified behaviors into three main types, each representing a different aspect of how your code operates:
+
+- **Happy Path**: Behaviors under this category represent the ideal and expected use cases of your code, where everything operates as intended without any errors or exceptions.
+- **Edge Case**: These are behaviors that occur at the boundaries of your code's logic, handling unusual or extreme inputs or scenarios that might not be immediately obvious.
+- **Other**: This catch-all category encompasses behaviors that do not neatly fit into the first two categories, including less common use cases or those that require special consideration.
+
+## Exploring Sub Behaviors
+Each identified behavior can be expanded to reveal its sub behaviors, which are more specific instances or variations of the main behavior. This allows for a granular understanding of how your code functions in different scenarios. To explore sub behaviors:
+
+1. **Click the Arrow**: Next to each behavior, an arrow icon allows you to expand the behavior and view its sub behaviors.
+2. **Review Sub Behaviors**: Each sub behavior represents a more explicit use case of the parent behavior, providing insight into detailed operational nuances of your code.
+
+## Generating Tests for Behaviors
+Codiumate not only identifies behaviors but also automatically generates tests for a selected set of initial behaviors to jumpstart your testing process. You have the flexibility to:
+
+- **Select More Behaviors**: Beyond the initial selection, you can choose additional behaviors or sub behaviors for which you want tests to be generated.
+- **Create Custom Behaviors**: If a specific behavior you're interested in is not listed, Codiumate allows you to define it manually:
+    1. **Add Behavior**: Enter a natural language description of the desired behavior in the "Add behavior" field.
+    2. **Generate Test**: Click the "Add and Generate" button to create a test based on your custom behavior description.
+
+By leveraging the behavior analysis and test generation capabilities of Codiumate, you can ensure comprehensive test coverage, capturing the full spectrum of how your code operates—from the most common scenarios to the edge cases that could lead to unexpected behavior.

docs/tests/configuration.md

@@ -0,0 +1,36 @@
+# Testing Configuration
+
+## Overview
+The Configuration tab within Codiumate's Advanced Panel provides a suite of options to tailor the test suite generation process to your specific needs. These configurations ensure that the generated tests align with your project's conventions, requirements, and preferences.
+
+## Configuration Options
+
+### 1. General Instructions
+A free text field where you can specify general instructions that apply to the entire test suite. Use this space to request specific styling, documentation inclusion, or any other overarching guidelines you'd like Codiumate to follow during test generation.
+
+### 2. Example Test
+Provide an example test in this field to guide Codiumate on your preferred naming conventions, styling, use of mocks, etc. Codiumate will analyze this example to align the generated tests with your project's existing patterns and practices.
+
+### 3. Number of Tests
+Set the desired number of tests for Codiumate to generate initially. This allows you to control the volume of tests based on your project's scale, coverage goals, or other considerations.
+
+### 4. Testing Framework
+Select your preferred testing framework from the available options. This ensures that the tests Codiumate generates are compatible with your project's testing environment and conventions.
+
+## Regenerate Test Suite
+
+After making any changes to the configurations, it's essential to apply these updates by clicking the **"Regenerate"** button. This action prompts Codiumate to regenerate the test suite based on the new configuration settings, ensuring that all modifications are accurately reflected in the generated tests.
+
+## Saving Configuration to a File
+
+To preserve your configurations and ensure consistency across your project or team, Codiumate allows you to save these settings to a TOML file:
+
+1. **Save Configurations**: Within the Configuration tab, find the option to export your settings.
+2. **File Name**: Save the exported configurations to a file named `.codiumai.toml`.
+3. **File Location**: It's recommended to place this file in the root folder of your project for easy access and recognition.
+4. **Version Control**:
+    - **Collaboration**: Consider adding `.codiumai.toml` to your project's git repository to share these configurations with your team, promoting uniformity in test generation across all contributors.
+    - **Local Preferences**: Alternatively, if the configurations are intended for personal use, you might opt to add `.codiumai.toml` to your `.gitignore` file to keep them local.
+
+By fine-tuning Codiumate's test generation through these configurations, you ensure that the output not only meets your technical requirements but also seamlessly integrates with your project's style and conventions.
+

docs/tests/context.md

@@ -0,0 +1,28 @@
+# Understanding Context in Test Generation
+
+## Overview
+Effective test generation in Codiumate relies heavily on understanding the context of your project. When initiating test generation, Codiumate meticulously collects context based on your code's dependencies and interactions. This rich context is crucial for generating accurate and meaningful tests that closely align with your project's specific requirements.
+
+## Context Collection
+Codiumate's context collection process encompasses several aspects:
+
+- **Code Dependencies**: Analyzes the dependencies within your code to understand the relationships and functionalities that need to be tested.
+- **Existing Tests**: Searches for existing tests within your project that could serve as templates for new tests, enhancing style consistency across your test suite. [Learn more about example tests](./example-test.md).
+
+## Reviewing Context
+To gain insights into the context Codiumate has collected for test generation:
+
+1. **Access the Context Tab**: The advanced panel features a "Context" tab where the collected context is displayed.
+2. **Explore Context Tags**: Each piece of context is tagged, providing a clear indication of its source and type.
+3. **Navigate to Source**: Clicking on a context tag will take you directly to the corresponding code or test, allowing you to review the basis for Codiumate's test generation decisions.
+
+## Context Types
+Each context tag is associated with a type, which helps in understanding the role it plays in test generation:
+
+- **Reference Test**: Existing tests in your project that provide styling cues or functional insights for generating new tests.
+- **Imported File**: Files that your code depends on, offering structural and functional context that influences test generation.
+
+## Benefits of Context Awareness
+By leveraging detailed context, Codiumate ensures that the generated tests are not only syntactically correct but also deeply integrated with the logic and architecture of your project, resulting in a robust and comprehensive test suite.
+
+Understanding the context behind test generation allows you to appreciate the depth of Codiumate's analysis and its impact on producing high-quality, relevant tests tailored to your project's unique environment.

docs/tests/example-test.md

@@ -0,0 +1,22 @@
+# Using Example Tests for Better Test Generation
+
+## Overview
+Codiumate enhances its test generation capabilities by allowing the use of example tests. This feature enables Codiumate to generate new tests that follow a specific style or format you prefer, ensuring consistency and adherence to your project's testing conventions.
+
+## Providing an Example Test
+There are several ways to provide Codiumate with an example test, each designed to suit different workflows and preferences:
+
+1. **Direct Input**: Copy and paste a test you like into the "Example Test" field located in the main test tab. Once you regenerate the test suite, Codiumate will attempt to generate new tests mirroring the styling of your example.
+2. **Configuration Tab**: For a more permanent setup, you can add the example test to the configuration tab. Saving this to the [configuration file](./configuration.md#saving-configuration-to-a-file) means all future tests generated by Codiumate will use this example as a template.
+3. **Automatic Detection**: If no example test is provided, Codiumate will search your project for a suitable test to use as an example. The identified test will be displayed in the main test tab with a note explaining Codiumate's choice. You have the option to delete or change this test if it doesn't meet your requirements.
+
+## Benefits of Using Example Tests
+
+- **Consistency**: Ensures that all generated tests adhere to your preferred style and formatting, maintaining consistency across your test suite.
+- **Customization**: Gives you the flexibility to influence the structure and content of the generated tests, tailoring them to better fit your project's needs.
+- **Efficiency**: Saves time by automating the generation of tests that align with your existing testing practices, reducing the need for manual adjustments.
+
+## Managing the Example Test
+
+- **Change or Remove**: At any point, you can modify or remove the current example test from the main test tab to adjust Codiumate's test generation behavior.
+- **Update Configuration**: To change the default example test for future test generations, simply update the example test in the configuration tab and save your changes.

docs/tests/how-to-use.md

@@ -1,5 +1,41 @@
-- lens
-- right click
-- command palette
-- codium button JB
-- chat
+# How to Use Test Generation in Codiumate
+
+Test Generation is a standout feature of Codiumate, designed to streamline and enhance the process of creating comprehensive test suites for your code. Unlike the interactive chat, test generation operates within a dedicated advanced panel, providing a focused environment for creating and managing tests.
+
+## Initiating Test Generation
+
+There are multiple pathways to initiate test generation, catering to your workflow and preferences:
+
+### From the Chat
+- **`/test-suite` Command**: Execute the `/test-suite` command on selected code within the chat interface to trigger test generation.
+
+### From the Editor
+1. **Codiumate Test Lens**: Click the "Codiumate: test" lens button appearing above every function, class, or method.
+2. **Context Menu**: Right-click on any selected code or component name and select "Codiumate - Test this component" from the context menu.
+3. **Command Palette in VSCode**: Highlight the desired code, open the command palette, and execute "Codiumate: Generate tests."
+4. **JetBrains IDE**: Click on the Codiumate icon located near every identifiable component.
+
+## The Advanced Panel
+
+Upon initiating test generation, the advanced panel opens, marking the beginning of your test creation process. The panel is divided into two main sections:
+
+### List of Behaviors
+The panel starts by presenting a list of behaviors detected in your selected code. Each behavior corresponds to a specific use case or functionality that your code is expected to perform, serving as a foundation for test coverage.
+
+**[Read more about behaviors](./behaviors.md)**
+
+### Generated Tests
+Following the behavior list, you will see generated tests covering a selection of the identified behaviors. These tests are designed to provide immediate value, offering examples of how each behavior can be tested.
+
+**[Read more about the generated tests](./test-suite.md)**
+
+### Example Test
+
+At the top of the tab, there's an Example Test field, that lets you guide the style of generated tests by providing a sample test. Insert a test in the designated field or save it in the settings. If unspecified, Codiumate chooses an existing project test as a template, ensuring consistency across your test suite.
+
+**[Read more about example test](./example-test.md)**
+
+
+## Next Steps
+
+After reviewing the generated tests and behaviors, you can further refine the tests, add new tests for uncovered behaviors, or modify existing tests to better suit your project's needs. The advanced panel provides a powerful, interactive environment for enhancing your project's test coverage and ensuring the reliability of your code.

docs/tests/index.md

@@ -1,3 +1,48 @@
-# Test Generation
+# Introduction to Test Generation in Codiumate
 
-Coming Soon!
+## Overview
+Codiumate revolutionizes the way developers create, refine, and manage test suites for their projects. By harnessing advanced AI technology, Codiumate supports test generation across **all programming languages**, making it a versatile tool for developers worldwide. This feature is designed to enhance productivity, improve code quality, and streamline the testing process, ensuring your software meets the highest standards of reliability and performance.
+
+## Key Features
+
+### Universal Language Support
+Codiumate proudly offers test generation capabilities for **all programming languages**, ensuring every developer can benefit from automated test creation. 
+
+**[Learn more about supported languages](./supported-languages.md).**
+
+## Initiating Test Generation
+Initiating test generation is straightforward, with multiple entry points to suit your workflow:
+
+- **From the Chat**: Utilize the `/test-suite` command.
+- **From the Editor**: Access Codiumate's features through lens buttons, context menus, or command palettes in supported IDEs like VSCode and JetBrains.
+- **The Advanced Panel**: All test generation activities converge in the advanced panel, where behaviors are analyzed, and tests are crafted.
+
+**[Learn more about using the advanced panel](./how-to-use.md)**.
+
+### Refining Your Test Suite
+
+Codiumate doesn't stop at generating tests. It offers an array of features for refining and customizing your test suite:
+
+- **Example Tests**: Guide the style of your generated tests by providing Codiumate with an example test.
+- **Context Awareness**: Codiumate gathers context from your code to generate relevant and accurate tests.
+- **Running and Auto-fixing Tests**: Validate and automatically fix your tests with Codiumate's run and auto-fix feature, available for certain supported languages.
+
+**[Learn more about test suite refinement](./test-suite.md)**.
+
+### Running Tests
+Codiumate integrates directly with your development environment to offer a streamlined process for running tests. This feature, available for Python, JavaScript, and TypeScript, enables you to:
+
+- **Run and Auto-Fix Tests**: Automatically run generated tests with the option for Codiumate to attempt fixes on failing tests, enhancing efficiency by iteratively improving test success rates.
+- **Manual Test Execution**: Choose to manually run tests without auto-fix interventions, providing you with direct insight into test outcomes.
+
+**[Learn more about running tests](./running-tests.md)**.
+
+### Configuration
+
+Tailor Codiumate's test generation to your project's needs through the Configuration tab, and continuously improve the test generation process by providing feedback on generated tests.
+
+**[Learn more about configuration](./configuration.md)**.
+
+## Conclusion
+
+Codiumate's test generation feature is a comprehensive solution designed to support developers in creating robust, reliable software. By automating and refining the test creation process, Codiumate not only saves valuable time but also enhances the overall quality of your projects.

docs/tests/running-tests.md

@@ -0,0 +1,37 @@
+# Running Tests in Codiumate (VSCode)
+
+## Overview
+For VSCode users working with [supported languages](./supported-languages.md#test-running-support), Codiumate offers an integrated feature to run tests directly from the advanced panel. This ensures your generated test suite performs as expected right off the bat. You have two primary options for running tests: with automatic fixes or without.
+
+## Running Tests Options
+
+1. **Run and Auto Fix**: This default option attempts to automatically fix any issues if a test fails. [Learn more about Run and Auto Fix](#run-and-auto-fix-flow).
+2. **Run Only**: Accessible by clicking the arrow next to the "Run and auto-fix" button, this option runs the test without attempting any automatic fixes.
+
+Both single tests and all tests in the suite can be executed using the "Run all tests" button located above the test list.
+
+## Run and Auto-Fix Flow
+
+When opting for "Run and auto-fix", Codiumate acts as a testing agent through the following steps:
+
+1. **Initial Test Run**: Codiumate runs the selected test(s).
+2. **Success Path**: If the test passes, the process concludes successfully.
+3. **Failure Analysis**:
+    - If a test fails, Codiumate analyzes the failure and displays a summary under the `POSSIBLE TEST ISSUE` section.
+    - **Test Issue**: If the issue lies within the test, Codiumate attempts a fix and reruns the test, looping back to step 1.
+    - **Code Issue**: If the analysis suggests a bug in the code, Codiumate halts auto-fixing and alerts the user. A bug summary and a "fix code" option are presented under the `POSSIBLE CODE ISSUE`. Clicking "fix code" generates a diff view with the proposed fix, which the user can accept to update the code and regenerate the test suite.
+
+## Handling Test Run Issues
+
+There might be instances where tests cannot run due to various reasons, such as:
+
+- **Framework Installation**: The testing framework might not be installed in your project, requiring setup before proceeding.
+- **Import Issues**: Problems with imports or dependencies can also prevent tests from running successfully.
+
+In such cases, Codiumate provides a summary of the issue without attempting auto-fixes, guiding you towards resolving the underlying problem to enable test execution.
+
+## Supported Languages
+
+This feature is available for a select range of languages supported by Codiumate in VSCode. [See the list of supported languages](./supported-languages.md#test-running-support).
+
+By leveraging Codiumate's testing features in VSCode, developers can ensure their test suites are not only comprehensive but also robust and error-free, streamlining the testing process and enhancing code quality.

docs/tests/supported-languages.md

@@ -0,0 +1,25 @@
+# Supported Languages in Codiumate
+
+Codiumate offers extensive support across a wide range of programming languages, ensuring developers can leverage its test generation capabilities regardless of their project's language. This inclusivity enhances Codiumate's utility, making it a versatile tool for developers seeking to improve their testing practices.
+
+## Test Generation Support
+
+Codiumate supports test generation for **ALL programming languages**, offering a broad spectrum of developers the ability to automatically create test suites for their code.
+
+### Enhanced Support for Specific Languages
+
+For **Python, JavaScript, TypeScript, and Java**, Codiumate goes a step further by supporting multiple testing frameworks.
+
+## Test Running Support
+
+While Codiumate's test generation feature is universally applicable, the ability to run tests directly from Codiumate is currently available for a select group of languages:
+
+- **Python**
+- **JavaScript**
+- **TypeScript**
+
+This functionality enables developers working in these languages to not only generate tests but also execute them within the Codiumate environment, providing immediate feedback on their efficacy and coverage.
+
+--- 
+
+This feature is available for VSCode users only.