Allow just one worker to be the background schedule leader (#836)

* Add a leader election mechanism to circumvent runtime issues for multiple schedulers

- Reduce the load on the DB and risk of issues on the service side by limiting the execution environment to one elected leader at a given time. This one is responsible for managing all of the execution of the jobs, though all workers are capable of adding and removing jobs

* Set a max duration for the schedule leader position (12 hrs), add some error if automation not added successfully

src/khoj/database/models/__init__.py

@@ -120,6 +120,7 @@ class ProcessLock(BaseModel):
     class Operation(models.TextChoices):
         INDEX_CONTENT = "index_content"
         SCHEDULED_JOB = "scheduled_job"
+        SCHEDULE_LEADER = "schedule_leader"
 
     # We need to make sure that some operations are thread-safe. To do so, add locks for potentially shared operations.
     # For example, we need to make sure that only one process is updating the embeddings at a time.