U 2ܟj*@sdZddlmZddlmZmZddlmZmZm Z m Z m Z ddl m Z mZmZddlmZddlmZmZddlmZdd lmZdd lmZe d Ze d ZGd ddeZGdddeZe Z!GdddZ"dS)z twisted.python.threadpool: a pool of threads to which we dispatch tasks. In most cases you can just use C{reactor.callInThread} and friends instead of creating a thread pool directly. ) annotations)Threadcurrent_thread)AnyCallableListOptionalTypeVar) ParamSpecProtocol TypedDict)pool)contextlog) deprecated)Failure)Version_P_Rc@seZdZddddZdS)_SupportsQsizeintreturncCsdSNselfrr=/tmp/pip-unpacked-wheel-4b529toj/twisted/python/threadpool.pyqsizesz_SupportsQsize.qsizeN__name__ __module__ __qualname__rrrrrrsrc@seZdZUded<ded<dS)_StaterminmaxN)r r!r"__annotations__rrrrr#"s r#c@s^eZdZdZdZdZdZdZdZe Z e e e dddd d d eZe eZd?d d d dddZed dddZeddddZeddddZeddddZeZddddZdddd Zd!dd"d#Zddd$d%Zd&dd'd(d)Zd&dd*d+Zd,d-d.dd/d0d1Zd2d3d-d.dd4d5d6Zddd7d8Z d@d9d9dd:d;d<Z!ddd=d>Z"dS)A ThreadPoola This class (hopefully) generalizes the functionality of a pool of threads to which work can be dispatched. L{callInThread} and L{stop} should only be called from a single thread. @ivar started: Whether or not the thread pool is currently running. @type started: L{bool} @ivar threads: List of workers currently running in this thread pool. @type threads: L{list} @ivar _pool: A hook for testing. @type _pool: callable compatible with L{_pool} FNZTwistedrzthreading.current_thread)version replacementrz Optional[str]) minthreads maxthreadsnamecsr|dkstd||ks td|_|_|_g_ddddfdd }d d fd d }||_d S)ac Create a new threadpool. @param minthreads: minimum number of threads in the pool @type minthreads: L{int} @param maxthreads: maximum number of threads in the pool @type maxthreads: L{int} @param name: The name to give this threadpool; visible in log messages. @type name: native L{str} rminimum is negativeminimum is greater than maximumrr)akwrcs(j|di|}j||S)Nr0) threadFactory _generateNamethreadsappend)r3r4threadrrrtrackingThreadFactory`s z2ThreadPool.__init__..trackingThreadFactoryrrcsjs dSjS)Nr)startedr%rrrr currentLimitgsz)ThreadPool.__init__..currentLimitN)AssertionErrorr$r%r0r7_pool_team)rr.r/r0r:r<rrr__init__JszThreadPool.__init__rcCs|j}|j|jS)a For legacy compatibility purposes, return a total number of workers. @return: the current number of workers, both idle and busy (but not those that have been quit by L{ThreadPool.adjustPoolsize}) @rtype: L{int} )r? statisticsidleWorkerCountbusyWorkerCount)rstatsrrrworkersns zThreadPool.workersz list[None]cCsdg|jjS)z For legacy compatibility purposes, return the number of busy workers as expressed by a list the length of that number. @return: the number of workers currently processing a work item. @rtype: L{list} of L{None} N)r?rArCrrrrworkingzs zThreadPool.workingcCsdg|jjS)a, For legacy compatibility purposes, return the number of idle workers as expressed by a list the length of that number. @return: the number of workers currently alive (with an allocated thread) but waiting for new work. @rtype: L{list} of L{None} N)r?rArBrrrrwaiterss zThreadPool.waitersrcsGfddd}|S)z For legacy compatibility purposes, return an object with a C{qsize} method that indicates the amount of work not yet allocated to a worker. @return: an object with a C{qsize} method. cseZdZddfdd ZdS)z$ThreadPool._queue..NotAQueuerrcs jjS)a Pretend to be a Python threading Queue and return the number of as-yet-unconsumed tasks. @return: the amount of backlogged work not yet dispatched to a worker. @rtype: L{int} )r?rAbackloggedWorkCount)qrrrrs z*ThreadPool._queue..NotAQueue.qsizeNrrrrr NotAQueuesrJr)rrJrrr_queues  zThreadPool._queueNonecCs4d|_d|_||jj}|r0|j|dS)z' Start the threadpool. FTN)joinedr;adjustPoolsizer?rArHgrow)rbacklogrrrstarts  zThreadPool.startcCs|jddS)z Increase the number of available workers for the thread pool by 1, up to the maximum allowed by L{ThreadPool.max}. r+N)r?rOrrrr startAWorkerszThreadPool.startAWorkerstrcCsd|jpt|d|jS)z Generate a name for a new pool thread. @return: A distinctive name for the thread. @rtype: native L{str} z PoolThread--)r0idrErrrrr6szThreadPool._generateNamecCs|jddS)zn Decrease the number of available workers by 1, by quitting one as soon as it's idle. r+N)r?shrinkrrrr stopAWorkerszThreadPool.stopAWorkerr#)statercCs"t|d|t||j|jdS)N__dict__)setattrr'r@r$r%)rrXrrr __setstate__s zThreadPool.__setstate__cCst|j|jdS)N)r$r%)r#r$r%rrrr __getstate__szThreadPool.__getstate__zCallable[_P, object]z_P.argsz _P.kwargs)funcargsr4rcOs|jd|f||dS)a Call a callable object in a separate thread. @param func: callable object to be called in separate thread @param args: positional arguments to be passed to C{func} @param kw: keyword args to be passed to C{func} N)callInThreadWithCallback)rr]r^r4rrr callInThreads zThreadPool.callInThreadz&Optional[Callable[[bool, _R], object]]zCallable[_P, _R])onResultr]r^r4rcsV|jr dStjjdddfdd fdd_|_|jdS) a$ Call a callable object in a separate thread and call C{onResult} with the return value, or a L{twisted.python.failure.Failure} if the callable raises an exception. The callable is allowed to block, but the C{onResult} function must not block and should perform as little work as possible. A typical action for C{onResult} for a threadpool used with a Twisted reactor would be to schedule a L{twisted.internet.defer.Deferred} to fire in the main reactor thread using C{.callFromThread}. Note that C{onResult} is called inside the separate thread, not inside the reactor thread. @param onResult: a callable with the signature C{(success, result)}. If the callable returns normally, C{onResult} is called with C{(True, result)} where C{result} is the return value of the callable. If the callable throws an exception, C{onResult} is called with C{(False, failure)}. Optionally, C{onResult} may be L{None}, in which case it is not called at all. @param func: callable object to be called in separate thread @param args: positional arguments to be passed to C{func} @param kw: keyword arguments to be passed to C{func} NrLrcsfz}d}Wntk r.t}d}YnXd_jdk rT||d_n|sbt|dS)NTF)theWork BaseExceptionrrarerr)resultok) inContextrrrh s   z6ThreadPool.callInThreadWithCallback..inContextcstjfSr)rcallr)r^ctxr]r4rrsz5ThreadPool.callInThreadWithCallback..) rMrZtheContextTrackerZcurrentContextZcontextsrcrar?Zdo)rrar]r^r4r)r^rjr]rhr4rr_s$z#ThreadPool.callInThreadWithCallbackcCs.d|_d|_|j|jD] }|qdS)z9 Shutdown the threads in the threadpool. TFN)rMr;r?quitr7join)rr9rrrstop$s   zThreadPool.stopz Optional[int])r.r/rcCs|dkr|j}|dkr|j}|dks,td||ksr@propertyrErFrGrKrIrQrRr6rWr[r\r`r_rnrNrprrrrr'*sT $     A r'N)#rq __future__r threadingrrtypingrrrrr Ztyping_extensionsr r r Ztwisted._threadsr r>Ztwisted.pythonrrZtwisted.python.deprecaterZtwisted.python.failurerZtwisted.python.versionsrrrrr#objectZ WorkerStopr'rrrrs