VERSION 1.0.0b1

[diogoncalves]

LLMstudio Version 1.0.0b1

What was done in this PR:

• new libraries: llmstudio-core, llmstudio-tracker, llmstudio-proxy, llmstudio (former monolith)
• Modularization and Refactoring: Shifted the structure towards modularization to enhance maintainability and scalability. Significant changes were made in the core components to make them more modular, including separating server components and organizing imports.
• LLM Provider Updates: Implemented changes in the LLM provider instantiation. The provider now requires additional parameters such as api_key and accepts a structured chat_request. This restructuring also includes making the proxy and tracking optional for more flexible usage.
• Feature Enhancements:
  • Added asynchronous chat support, both with and without streaming, and implemented synchronous methods.
  • Introduced session_id tracking for chat sessions and logging support for better traceability.
  • Incorporated dynamic API versioning and the ability to handle multiple API providers, including OpenAI, Azure, and VertexAI.
  • Added support for LangChain agents and tool calling with parallel execution.
  • Adapted VertexAI integration for both regular and tool-calling functions.
  • Provided support for legacy UTC compatibility to address compatibility with older Python versions.
• Automated Testing and Documentation:
  • Enhanced test automation for development processes, including adding tests for modularized components.
  • Updated the documentation, added docstrings to LLM classes, and provided a tutorial on using langgraph.
• Formatting and Configuration:
  • Applied consistent formatting, added isort and flake8 configurations, and updated pre-commit hooks.
  • Updated config.yaml for easier configuration management and enhanced initialization for Azure-based clients.

How it was tested:

• Conducted automated tests across development environments to verify the correctness of modularized components and new functionality.
• Validated the functionality of asynchronous and synchronous chat capabilities with various LLM providers, including OpenAI, Azure, and VertexAI.
• Tested new session_id integration for accuracy in tracking sessions and seamless functionality with logging tools.
• Verified compatibility across Python versions, particularly with UTC handling in legacy versions.
• Reviewed server component separation and ensured compatibility with modular server deployments.

Additional notes:

• Any breaking changes?
  Yes, breaking changes were introduced in the LLM provider instantiation. The new instantiation now requires specifying parameters in the following format:

  llm = LLM(provider=provider, api_key=api_key, **kwargs)
  chat_request = {
      "chat_input": "Hello, my name is Json",
      "model": model,
      "is_stream": False,
      "retries": 0,
      "parameters": {
          "temperature": 0,
          "max_tokens": 100,
          "response_format": {"type": "json_object"},
          "functions": None,
      }
  }
  llm.achat(**chat_request)

• Any new dependencies added?
  New dependencies are introduced to support modularization, API versioning, and enhanced testing, though specific package details should be checked in pyproject.toml.

• Any performance improvements?
  Optimized API calls with parallel execution for tool calls, streamlined server components, and minimized function calls within the LLM provider, which should contribute to overall performance improvements.

To fix the problem, we need to bind the socket to a specific interface instead of all interfaces. This can be achieved by replacing the empty string ("") with a specific IP address. If the specific IP address is not known or needs to be configurable, we can use an environment variable to set it.

1. Modify the assign_port function to accept an additional parameter for the IP address.
2. Use this IP address to bind the socket instead of the empty string.
3. Update the defaults dictionary to include a default IP address.
4. Ensure the IP address is read from the environment variables.

To fix the problem, we need to bind the socket to a specific interface instead of all interfaces. This can be achieved by using a dedicated IP address for the interface we want to bind to. If the specific interface is not known or needs to be configurable, we can use an environment variable to specify the interface.

1. Modify the assign_port function to accept an additional parameter for the interface IP address.
2. Use this parameter to bind the socket to the specified interface instead of all interfaces.
3. Update the code to use a default interface IP address if none is provided.

To fix the problem, we should bind the socket to a specific interface rather than all interfaces. This can be achieved by replacing the empty string ("") with a specific IP address. In this case, we will use localhost (i.e., 127.0.0.1) to restrict the socket to the local machine. This change will ensure that the socket only accepts connections from the local machine, thereby enhancing security.

To fix the problem, we need to bind the socket to a specific network interface instead of all interfaces. This can be achieved by replacing the empty string "" with a specific IP address. In this case, we will use the LLMSTUDIO_TRACKING_HOST environment variable, which defaults to localhost. This ensures that the socket is only accessible from the local machine, mitigating the security risk.

1. Modify the assign_port function to accept an additional parameter for the host.
2. Use the TRACKING_HOST environment variable as the host when binding the socket.

To fix the problem, we need to bind the socket to a specific interface instead of all interfaces. This can be achieved by replacing the empty string "" with a specific IP address, such as 127.0.0.1 (localhost), which limits the socket to accept connections only from the local machine.

• Update the assign_port function to bind the socket to 127.0.0.1 instead of "".
• Ensure that the changes do not affect the existing functionality of dynamically assigning a port.

To fix the problem, we need to bind the socket to a specific network interface instead of all interfaces. This can be achieved by replacing the empty string ("") with a specific IP address. In this case, we will use localhost (127.0.0.1) to bind the socket to the local interface, which is a common practice for services that do not need to be accessed from outside the host machine.

• Update the assign_port function to bind the socket to 127.0.0.1 instead of an empty string.
• Ensure that the socket is only accessible from the local machine, enhancing security.