Debanjum Singh Solanky e47922e53a Aggregate webpage extract queries to run once for each distinct webpage
This should reduce webpage read and response generation time.

Previously, we'd run separate webpage read and extract relevant
content pipes for each distinct (query, url) pair.

Now we aggregate all queries for each url to extract information from
and run the webpage read and extract relevant content pipes once for
each distinct url.

Even though the webpage content extraction pipes were previously being
in parallel. They increased response time by
1. adding more context for the response generation chat actor to
   respond from
2. and by being more susceptible to page read and extract latencies of
   the parallel jobs

The aggregated retrieval of context for all queries for a given
webpage could result in some hit to context quality. But it should
improve and reduce variability in response time, quality and costs.
2024-10-17 17:40:49 -07:00
2024-10-10 18:07:30 -07:00
2024-10-10 18:07:30 -07:00

Khoj Logo

test docker pypi discord

Your AI second brain

📑 Docs   •   🌐 Web   •   🔥 App   •   💬 Discord   •   ✍🏽 Blog


Khoj is a personal AI app to extend your capabilities. It smoothly scales up from an on-device personal AI to a cloud-scale enterprise AI.

  • Chat with any local or online LLM (e.g llama3, qwen, gemma, mistral, gpt, claude, gemini).
  • Get answers from the internet and your docs (including image, pdf, markdown, org-mode, word, notion files).
  • Access it from your Browser, Obsidian, Emacs, Desktop, Phone or Whatsapp.
  • Create agents with custom knowledge, persona, chat model and tools to take on any role.
  • Automate away repetitive research. Get personal newsletters and smart notifications delivered to your inbox.
  • Find relevant docs quickly and easily using our advanced semantic search.
  • Generate images, talk out loud, play your messages.
  • Khoj is open-source, self-hostable. Always.
  • Run it privately on your computer or try it on our cloud app.

See it in action

demo_chat

Go to https://app.khoj.dev to see Khoj live.

Full feature list

You can see the full feature list here.

Self-Host

To get started with self-hosting Khoj, read the docs.

Contributors

Cheers to our awesome contributors! 🎉

Made with contrib.rocks.

Interested in Contributing?

We are always looking for contributors to help us build new features, improve the project documentation, or fix bugs. If you're interested, please see our Contributing Guidelines and check out our Contributors Project Board.

Description
No description provided
Readme AGPL-3.0 116 MiB
Languages
Python 51%
TypeScript 36.1%
CSS 4.1%
HTML 3.2%
Emacs Lisp 2.4%
Other 3.1%