Scheduler causes pthread_mutex_lock assertion

[Peter]

I have code with something like this

Scheduler scheduler(n);
throw std::runtime_error("aha");

which causes the following assertion after catching the exception:

/usr/include/boost/thread/pthread/mutex.hpp:50: void boost::mutex::lock(): Assertion `!pthread_mutex_lock(&m)' failed

[Peter]

I guess the solution is to catch exception in the scope of the Scheduler. If that's the case, this is merely a documentation issue (and test).

[Peter]

I found this ​stackoverflow post, which refers to ​boost thread.

[Peter]

The answer in this thread pretty much describes how we want to use boost thread

​http://stackoverflow.com/questions/1374302/c-boost-code-example-of-throwing-an-exception-between-threads

The main question is how we want the scheduler to behave when an exception is detected?

It's probably a bad idea to rethrow immediately, because then the problem from description remains. I suppose we should make the workers return and we should fasten this as much as possible, either by interupt or by emptying the queue and give them a poisson pill. If going by the latter, that combination should probably be provided as a public member function as well.

In theory that means that multiple workers might throw before Scheduler rethrow, but I guess the subsequenth exceptions could be ignored and only act on the first one.

[Peter]

Outline: when a Worker catches an exception, assign a variable boost::exception_ptr (either member in Worker or Job) and exit. The Scheduler detects the exception, calls new public member function Scheduler::interrupt(void) and then boost::rethrow_exception(exception_ptr). The function Scheduler::interrupt(void) simply calls group_.interrupt_all() which they wait for each Worker to finish current Job and reach interruption_point (unless the current Job has interruption points within its operator() execution. That means that we allow all Workers/threads to exit before we throw from Scheduler.

[Peter]

(In [3405]) fixes #832