Limits and Considerations

Tokens

Each iDialogue tier has a metered number of AI “tokens” that can be used within a 24-hour period.

When allocated tokens are exceeded, end users may experience a “Max AI tokens exceeded (24-hour limit 100,000)” error message.

GPT3.5 fits conversations within a 16,000 token limit, or about 12,000 words.

Use of the GPT-4 model may incur additional costs (not available in Starter/Pro editions).

iDialogue Enterprise edition does not have a token limit. Overrages are billed separately. Or the Subscriber may configure their own OpenAI API key.

Prompt Size

The size of flow-generated prompt is limited to Salesforce Flow output variable max size (currently around 64Kb).

Performance

”Smaller is faster” is the best general advice.

Initial requests in a chat session incur some initialization performance costs. Subsequent requests may cache chat sessions for up to 24 hours.

AI Assistants

There is no limit to the number of AI Assistant modes deployed for an SObject. However, the mode buttons may spill over and not render as desired.

Same limits apply to sidebar placement of the AI Assistant on page layouts.

File GPT

Files up to 2GB may be indexed for conversational AI. Larger files will have slower search performance.

File GPT bots may only return useful results that come directly from the system prompt or ContentVersion record data.

File GPT bots may not have long conversational memory relative to SObject bots. File GPT bots reserve their token limits for document text, and not past chat history.

Room ChatBots

AI Models can be developed today based on the “Room” SObject type (rooms__DocumentRoom__c) for use in customer-facing chats. However, enabling customer-facing chats is currently limited to customers using custom domains at this time.

Broader deployment of customer-facing models should contact info@idialogue.app and align development with moderation features that are in pilot.