U Tg@@s&dZddlZddlZddlmZddlmZddlmZm Z m Z m Z m Z m Z mZmZmZmZddlmZmZddlmZmZmZddlmZdd lmZmZmZdd lm Z m!Z!dd l"m#Z#dd l$m%Z%dd l&m'Z'ddl(m)Z)ddl*m+Z+ddl,m-Z-ddl.m/Z/m0Z0ddl1m2Z2ddl3m4Z4ddl5m6Z6m7Z7ddl8m9Z9m:Z:ddl;mZ>m?Z?m@Z@mAZAddlBmCZCmDZDmEZEGddde e!ZFeFdddZGeGd d!d!ZHeGd"d#d#ZIeGd$d%d%ZJeKe/eKd&d'd(ZLeIe e0eMs   r>c@sneZdZUdZeeed<eed<eed<e e ed<e e ed<e Z eeddd d Zdd d d ZdS)StartedWorkerPoolaE A pool of workers which have already been started. @ivar workingDirectory: A directory holding the working directories for each of the workers. @ivar testDirLock: An object representing the cooperative lock this pool holds on its working directory. @ivar testLog: The open overall test log file. @ivar workers: Objects corresponding to the worker child processes and adapting between process-related interfaces and C{IProtocol}. @ivar ampWorkers: AMP protocol instances corresponding to the worker child processes. r@ testDirLocktestLogworkers ampWorkersN) workerActionr9cs"tfdd|jDIdHdS)zB Run an action on all of the workers in the pool. c3s|]}t|VqdSN)r+.0workerrKr6r7 sz(StartedWorkerPool.run..N)rrJ)selfrKr6rPr7runs zStartedWorkerPool.runr8cs~tdd|jDddIdH}t|D]&\}\}}|s&|jd|d|q&|jdd=|jdd=|j|j dS)zv Shut down all of the workers in the pool. The pool is unusable after this method is called. cSsg|]}t|qSr6)r fromCoroutineexitrMr6r6r7 sz*StartedWorkerPool.join..T)Z consumeErrorsNzjoining disttrial worker #z failed) rrI enumerate_loggerfailurerJrHcloserGZunlock)rRresultsnZ succeededrYr6r6r7joins    zStartedWorkerPool.join)r2r3r4r5rrrDrr rr.r/rrXr0rSr]r6r6r6r7rFcs     rFc@sReZdZUdZeed<eeee e e e dddZ ddZeedd d Zd S) WorkerPoolz Manage a fixed-size collection of child processes which can run tests. @ivar _config: Configuration for the precise way in which the pool is run. _config) protocolsr@rBr9csfddt|DS)a Create local worker protocol instances and return them. @param protocols: The process/protocol adapters to use for the created workers. @param workingDirectory: The base path in which we should run the workers. @param logFile: The test log, for workers to write to. @return: A list of C{quantity} C{LocalWorker} instances. cs&g|]\}}t|t|qSr6)r.childrE)rNxprotocolrBr@r6r7rVsz2WorkerPool._createLocalWorkers..)rW)rRr`r@rBr6rdr7_createLocalWorkerss zWorkerPool._createLocalWorkersc Csvtdjj}ddddddtdtdi}tj}tj t j|d<|D],}t j |g}| |||t j |||dqDd S) a Spawn processes from a list of process protocols. @param spawner: A C{IReactorProcess.spawnProcess} implementation. @param protocols: An iterable of C{ProcessProtocol} instances. @param arguments: Extra arguments passed to the processes. ztwisted.trial._dist.workertrialrwr&rr PYTHONPATH)argschildFDsenvN) rZfilePathpathr'r(osenvironcopypathsepr]sys executableextend) rRZspawnerr` argumentsZworkertrialPathrjrnrOrir6r6r7_launchWorkerProcessess$    z!WorkerPool._launchWorkerProcesses)r:r9cst|jj\}}t|jjr*t|jj}n||jj}t|}ddt|jj D}| |||}| |j ||jj t|||||S)z Launch all of the workers for this pool. @return: A started pool object that can run jobs using the workers. cSsg|] }tqSr6)r/)rNrbr6r6r7rVsz$WorkerPool.start..)r$r_r@rrBrZ preauthChildr%ranger?reruZ spawnProcessrArF)rRr:ZtestDirrGZ testLogPathrHrJrIr6r6r7starts2 zWorkerPool.startN)r2r3r4r5r>rDrr/rrr rr.rerurrFrwr6r6r6r7r^s  r^) untilFailureresultr9cCs |o |S)z Determine whether the test suite should be iterated again. @param untilFailure: C{True} if the suite is supposed to run until failure. @param result: The test result of the test suite iteration which just completed. ) wasSuccessful)rxryr6r6r7shouldContinues r{)pool testCasesry driveWorkerr9csJz|t|||IdHWn(tk rD|jtdtYnXdS)Nz )rSr ExceptionoriginaladdErrorr#r)r|r}ryr~r6r6r7runTestssrc@sfeZdZUdZeZeZede fe d<e e d<e e e d<dZee d<edeed d Zee d <edeejd Zee d <d Ze e d<dZee d<dZee d<dZe e d<dZe e d<eZee gefe d<edddZ!ddZ"ee#e$e%ddddZ&d)e'e(e)feeddd Z*e'e(e)fee d!d"d#Z+e'e(e)fe d$d%d&Z,e'e(e)fe d$d'd(Z-dS)*DistTrialRunnera A specialized runner for distributed trial. The runner launches a number of local worker processes which will run tests. @ivar _maxWorkers: the number of workers to be spawned. @ivar _exitFirst: ``True`` to stop the run as soon as a test case fails. ``False`` to run through the whole suite and report all of the results at the end. @ivar stream: stream which the reporter will use. @ivar _reporterFactory: the reporter class to be used. ._reporterFactory _maxWorkers_workerArgumentsF _exitFirstN)factory)default converter_reactorstreamr_tracebackFormat_realTimeErrors_uncleanWarningsztest.log_logfileZ _trial_temp_workingDirectory_workerPoolFactoryr8cCs.|j|j|j|jd}|jr$t|}||S)zL Make reporter factory, and wrap it with a L{DistReporter}. )Zrealtime)rrrrrr"_distReporterFactory)rRreporterr6r6r7 _makeResultJszDistTrialRunner._makeResultcCs |dS)z Write test run final outcome to result. @param result: A C{TestResult} which will print errors and the summary. N)done)rRryr6r6r7 writeResultsUszDistTrialRunner.writeResults)ryr}rOr9cs*fdd}|D]}||IdHqdS)a Drive a L{LocalWorkerAMP} instance, iterating the tests and calling C{run} for every one of them. @param worker: The L{LocalWorkerAMP} to drive. @param result: The global L{DistReporter} instance. @param testCases: The global list of tests to iterate. @return: A coroutine that completes after all of the tests have completed. cs@z|IdHWn$tk r:j|tYnXdSrL)rSrrrr)caseryrOr6r7taskqsz*DistTrialRunner._driveWorker..taskNr6)rRryr}rOrrr6rr7 _driveWorker]szDistTrialRunner._driveWorker)suiterxr9cstt|tttjtjj j }j d| d|jIdHtt}tttdfdd }zt||IdHWSIdHXdS)a* Spawn local worker processes and load tests. After that, run them. @param suite: A test or suite to be run. @param untilFailure: If C{True}, continue to run the tests until they fail. @return: A coroutine that completes with the test result. zRunning z tests. N)r\r9csjrjd|ddjr8fdd}ndd}tt|jIdHS)Nz Test Pass r& cs jSrL)rrz_ryr6r7z@DistTrialRunner.runAsync..runAndReport..cSsdSNTr6rr6r6r7rr)rwriterrrr-rr)r\ZcasesConditionrRZ startedPoolr}rxrr7 runAndReports  z.DistTrialRunner.runAsync..runAndReport)listrrr>minlenrrrrrrrZcountTestCasesrwrrr{r*rCr)r]r,)rRrrxZ poolStarter conditionrr6rr7runAsynczs"   zDistTrialRunner.runAsync)testrxr9csddtttfddfdd }tttdfdd }ttdfd d }jd d |t || | |j t tr t tstd ttjS)NF)rgr9cs|dSrLr6)rgrr6r7capturesz%DistTrialRunner._run..capturer8csddkrSdSr)cancelr6)reactorStoppingrytestsInProgressr6r7maybeStopTestss z,DistTrialRunner._run..maybeStopTests)ryr9cssj|SrL)rstopr)rrRr6r7maybeStopReactors z.DistTrialRunner._run..maybeStopReactorbeforeshutdownz is not DistReporter)r rr)r robjectrZaddSystemEventTriggerrTrZaddBothrS isinstanceZraiseExceptionAssertionErrorr r r)rRrrxrrrr6)rryrRrr7_runs"  zDistTrialRunner._run)rr9cCs|j|ddS)za Run a reactor and a test suite. @param test: The test or suite to run. FrxrrRrr6r6r7rSszDistTrialRunner.runcCs|j|ddS)z| Run the tests with local worker processes until they fail. @param test: The test or suite to run. Trrrr6r6r7runUntilFailureszDistTrialRunner.runUntilFailure)F).r2r3r4r5r)rrrXrr rDrCrrErboolrrr=rr1rqstdoutrr rrrrrr^rr>rrr r!r/rr rrrrrSrr6r6r6r7r!sD             L,r)Or5rmrq functoolsros.pathrtypingrrrrrr r r r r ZunittestrrattrsrrrZattrs.convertersrZtwisted.internet.deferrrrZtwisted.internet.interfacesrrZtwisted.loggerrZtwisted.python.failurerZtwisted.python.filepathrZtwisted.python.lockfilerZtwisted.python.modulesrZ _asyncrunnerrZitrialr r!rr"runnerr#utilr$r%r'r(Z distreporterr)Z functionalr*r+r,r-rOr.r/r0r1r=r>rFr^rr{rrr6r6r6r7sV  0           8e