Enable free tier users to have unlimited chats with the default chat model (#886)

- Allow free tier users to have unlimited chats with default chat model. It'll only be rate-limited and at the same rate as subscribed users
- In the server chat settings, replace the concept of default/summarizer models with default/advanced chat models. Use the advanced models as a default for subscribed users.
- For each `ChatModelOption' configuration, allow the admin to specify a separate value of `max_tokens' for subscribed users. This allows server admins to configure different max token limits for unsubscribed and subscribed users
- Show error message in web app when hit rate limit or other server errors
This commit is contained in:
sabaimran
2024-08-16 12:14:44 -07:00
committed by GitHub
parent 8dad9362e7
commit c0316a6b5d
11 changed files with 210 additions and 92 deletions

View File

@@ -222,7 +222,20 @@ export default function Chat() {
try {
await readChatStream(response);
} catch (err) {
console.log(err);
console.error(err);
// Retrieve latest message being processed
const currentMessage = messages.find((message) => !message.completed);
if (!currentMessage) return;
// Render error message as current message
const errorMessage = (err as Error).message;
currentMessage.rawResponse = `Encountered Error: ${errorMessage}. Please try again later.`;
// Complete message streaming teardown properly
currentMessage.completed = true;
setMessages([...messages]);
setQueryToProcess("");
setProcessQuerySignal(false);
}
}