#896 closed discussion (fixed)

Scheduler job processing

Reported by: peter Owned by: peter
Priority: major Milestone: yat 0.15
Component: utility Version: 0.14.3
Keywords: Cc:


When using Scheduler and Jobs have dependencies, the Scheduler processes completed tasks and if a Job is ready to be executed it is sent to the queue for the workers to chew on. In the current implementation this processing happens only when any of the functions


is called. In other words, if we are busy in the main thread doing something else, which might e.g. wait for a certain job to finish, we might get stuck because the processing when the Scheduler sends jobs to the job queue never happens.

Question is if this should be changed so this processing should be automatically done in a separate thread. Alternatively, introduce a new function that starts the processing in separate thread.

It is not obvious how this would interact with function ::wait(void).

Change History (2)

comment:1 Changed 11 months ago by peter

  • Milestone changed from yat 0.x+ to yat 0.15
  • Owner changed from jari to peter
  • Status changed from new to assigned

comment:2 Changed 11 months ago by peter

  • Resolution set to fixed
  • Status changed from assigned to closed

(In [3681]) closes #896

Reimplement Scheduler. It now spawns a specific thread to handle the jobs (see class JobHandler?). It means that the processing that previously was done in Schedyuler::wait is done is this thread instead and wait is now free of processing. The redesign has the side effect that functions like Scheduler::submit and Scheduler::add_dependency are more thread safe (more analysis warranted). Otherwise the behaviour is virtually unchanged, except that processing of completed (checking if their dependants are ready to queued) now starts running immediately rather than only running when a Scheduler function was called (::submit, ::add_dependancy or finally ::wait).

Note: See TracTickets for help on using tickets.