Develop AI applications with
your own private AI system

Frequently
asked
questions

• What is the difference between the 8B and 70B models?

  The two models differ significantly in size, performance, and parameter count. Parameters are like the “brain cells” of an AI model; the more parameters a model has, the more complex its reasoning, language understanding, and generation capabilities.

  The Llama 3.1-8B model is perfect for everyday interactions and simpler automation tasks, such as chatbots. With 8 billion parameters, it delivers fast, precise responses – ideal for companies focused on efficiency and cost-effectiveness.

  The Llama 3.1-70B model, with 70 billion parameters, provides even more detailed and precise answers, making it better suited for advanced analysis and complex applications, though it does require more computing power and resources.

• What components are included in the Managed LLM?

  The Managed LLM includes all essential elements for a comprehensive AI system:
  a powerful NVIDIA A100 or H100 GPU, the Large Language Model (LLM) Llama 3.1-8B or 70B, a user-friendly Chat GUI, and a REST API.

• What is a REST API?

  An API is a standard interface that enables different software applications to communicate and exchange data. A REST API is a specific type of API that standardizes and simplifies this communication.

  It allows the Managed LLM to integrate seamlessly into existing enterprise systems, such as ERP, accounting, or ticketing systems, making it an ideal addition for automating and streamlining numerous business processes.

• How many requests can be made simultaneously to the Managed LLM?

  The 8B model allows up to 10 concurrent requests, each with up to 16,000 tokens. Additionally, up to 512 further requests can be queued, which are then automatically processed in sequence. Average response times range between 0.5 and 5 seconds.

• What happens if the queue of 512 requests is full?

  If more than 512 simultaneous requests are made, the 513th request will receive an error message to ensure system performance.

  The first 512 requests are processed sequentially, and request 513 can be resubmitted after a few seconds.

  This ensures smooth operation and high responsiveness, even at maximum capacity.

• What are tokens?

  Tokens are small building blocks into which text is divided for language models to process.

  A token represents a text unit of about 0.75 words and can include combinations of syllables, punctuation, numbers, and/or spaces.

• What security standards does the Managed LLM meet?

  Your AI system runs in a dedicated Private Cloud in our own certified data center in Germany, meeting the international security standard TIER 3 and operating according to European and German data protection regulations.

  We handle software provisioning and maintenance ourselves, without relying on rented resources or third-party services. To protect your data, we use advanced security measures like firewalls, two-factor authentication, and an optional VPN connection. Encrypted data transmission is handled via secure APIs or VPN connections, with no data storage between requests and responses.

  For even more efficient and secure network connections, we offer an optional direct connection via SD-WAN, enabling a stable and secure link without the need for VPN.

  All Managed LLM transaction processing occurs within Germany, ensuring additional security and compliance with German data protection standards.

• What does support cover?

  Our 24/7/365 support provides you with continuous access to the AI system, ensuring 99.5% availability of the server, API, and LLM.

  During onboarding, we provide all essential information to make getting started as easy as possible. However, application consulting is not included.

• Are updates for the LLM provided?

  As a standard, updating the LLM to a newer or different version is not included in the service. However, this can be arranged upon request and for an additional fee.

• How is billing handled, and what is the contract term?

  Billing is conducted quarterly in advance at a fixed, usage-independent rate. The minimum contract term is 12 months.