1fd6e16cff
### **feat(obsidian): Enhance Sync Experience with Progress Bars and Bug Fixes** This pull request significantly improves the content synchronization experience for Obsidian users by fixing a critical bug and introducing new UI elements for better feedback and monitoring. The previous implementation could fail with `403 Forbidden` errors when syncing a large number of files due to server-side rate limiting. This update addresses that issue and provides users with clear, real-time feedback on storage usage and sync progress. --- ### Key Changes * **Improve Sync Robustness** Refactor `updateContentIndex` to sync files prioritized by file type (md > pdf > image) and batched by size (10Mb) and item limits (50 items). This respects server rate limits and ensures that large vaults can be indexed reliably without triggering `403` errors. * **Show Cloud Storage Usage Bar** A progress bar has been added to the settings page to display cloud storage usage. * **Total Limit**: The storage limit (**10 MB** for free, **500 MB** for premium) is now reliably determined by the `is_active` flag returned from the `/api/v1/user` endpoint, eliminating fragile client-side heuristics. * **Used Space**: The used space is calculated via a **client-side estimation** of all files configured for synchronization. This provides a clear and immediate indicator of the vault's storage footprint. * **Show Real-time Sync Progress Bar** When a manual sync is triggered via the "Force Sync" button, a progress bar now appears, providing real-time feedback on the operation. * It displays the number of files processed against the total number of files to be indexed or deleted. * This is implemented using a **callback mechanism** (`onProgress`) to cleanly communicate progress from the sync logic (`utils.ts`) to the UI (`settings.ts`) without coupling them. * **Auto-refresh Storage Used After Sync** The Cloud Storage Usage bar is now automatically refreshed upon the completion of a "Force Sync". This ensures the user immediately sees the updated storage estimation without needing to reopen the settings panel. --- ### Visuals <img width="980" height="237" alt="image" src="https://github.com/user-attachments/assets/2b3ce420-766b-476f-9fc0-c6b38c0226fb" /> --------- Co-authored-by: Debanjum <debanjum@gmail.com>
🎁 New
- Start any message with
/researchto try out the experimental research mode with Khoj. - Anyone can now create custom agents with tunable personality, tools and knowledge bases.
- Read about Khoj's excellent performance on modern retrieval and reasoning benchmarks.
Overview
Khoj is a personal AI app to extend your capabilities. It smoothly scales up from an on-device personal AI to a cloud-scale enterprise AI.
- Chat with any local or online LLM (e.g llama3, qwen, gemma, mistral, gpt, claude, gemini, deepseek).
- Get answers from the internet and your docs (including image, pdf, markdown, org-mode, word, notion files).
- Access it from your Browser, Obsidian, Emacs, Desktop, Phone or Whatsapp.
- Create agents with custom knowledge, persona, chat model and tools to take on any role.
- Automate away repetitive research. Get personal newsletters and smart notifications delivered to your inbox.
- Find relevant docs quickly and easily using our advanced semantic search.
- Generate images, talk out loud, play your messages.
- Khoj is open-source, self-hostable. Always.
- Run it privately on your computer or try it on our cloud app.
See it in action
Go to https://app.khoj.dev to see Khoj live.
Full feature list
You can see the full feature list here.
Self-Host
To get started with self-hosting Khoj, read the docs.
Enterprise
Khoj is available as a cloud service, on-premises, or as a hybrid solution. To learn more about Khoj Enterprise, visit our website.
Frequently Asked Questions (FAQ)
Q: Can I use Khoj without self-hosting?
Yes! You can use Khoj right away at https://app.khoj.dev — no setup required.
Q: What kinds of documents can Khoj read?
Khoj supports a wide variety: PDFs, Markdown, Notion, Word docs, org-mode files, and more.
Q: How can I make my own agent?
Check out this blog post for a step-by-step guide to custom agents. For more questions, head over to our Discord!
Contributors
Cheers to our awesome contributors! 🎉
Made with contrib.rocks.
Interested in Contributing?
Khoj is open source. It is sustained by the community and we’d love for you to join it! Whether you’re a coder, designer, writer, or enthusiast, there’s a place for you.
Why Contribute?
- Make an Impact: Help build, test and improve a tool used by thousands to boost productivity.
- Learn & Grow: Work on cutting-edge AI, LLMs, and semantic search technologies.
You can help us build new features, improve the project documentation, report issues and fix bugs. If you're a developer, please see our Contributing Guidelines and check out good first issues to work on.