o Ďi@sjdZdgZddlZddlmZddlmZddlm Z ddl Z ddl Z ddl Z ddl mZmZmZmZddlmZdd lmZdd lmZdd lmZdd lmZdd lmZddlmZddlm Z ddl!m"Z"ddl!m#Z#ddl!m$Z$ddl!m%Z%ddl!m&Z&ddl!m'Z'ee(e fZ)e*e(ee&fZ+e*e(e,e"fZ-e*e(ee&fZ.e/dee/dee/deGddde j0Z1GdddZ2dede(de j3e+d dfd!d"Z4 dMded#e j3e+d$e5d%ee j3e-d df d&d'Z6d(ed%e j3e-d)e j3e.d dfd*d+Z7d(ed#e j3e.d$e5d,e8d df d-d.Z9de(d/ee(d0ee(d1ee:d2ee:d$e5d dfd3d4Z;  dNd5ed6e,e(d7e5e(e5fd8e5e(e5fd9e5e(e5fd:e5e(e5fd;ee)dZ=  dNd6e,e(d?e5e(e5fd:e5e(e5fd@ed;ee)ddCed e*e1e jns$  ? r>detectorfilepathdetector_queuer cCs>t|}z||}|||fWdS||df)aqPrepares the input for detector inference. Responsible for loading and preprocessing an image in preparation for the detector. Args: detector: SpeciesNetDetector to use. filepath: Path to image to load and preprocess. detector_queue: Output queue for preprocessed images for detector inference. N)r preprocessput)rWrXrYimgr/r/r0_prepare_detector_inputs r] input_queue results_dict bboxes_queuecCsP|\}}|||}|||<|r&|dg}||dd|DfdSdS)a8Runs detector inference. Takes a preprocessed image from the input queue and runs the detector model on it. The raw output of the detector is stored in `results_dict` and the list of bounding boxes is additionally stored in `bboxes_queue`. Args: detector: SpeciesNetDetector to run. input_queue: Input queue of preprocessed images. results_dict: Output dict for inference results. bboxes_queue: Output queue for bounding boxes identified during inference. Optional. detectionscSg|]}t|dqSbboxr.0detr/r/r0 z!_run_detector..N)getpredictr[)rWr^r_r`rXr\ predictionrar/r/r0 _run_detectors   rm classifierclassifier_queuecCsN|\}}t|}z|j||d}|||fWdS||df)anPrepares the input for classifier inference. Takes bounding box information from `bboxes_queue` and uses it to load and preprocess the image for classifier inference. It first loads the image, preprocesses it by potentially cropping based on bboxes and finally outputs the image into `classifier_queue` to be used by the classifier model. Args: classifier: SpeciesNetClassifier to use. bboxes_queue: Input queue of bounding boxes from detector inference. classifier_queue: Output queue for preprocessed images for classifier inference. bboxesN)rjrrZr[)rnr`rorXrqr\r/r/r0_prepare_classifier_inputs rr batch_sizec s^fddt|D}dd|D}dd|D}|||}t||D]\}} | ||<q$dS)aRuns classifier inference. Takes a batch of preprocessed images from the input queue and runs the classifier model on it. The output of the classifier is stored in `results_dict`. Args: classifier: SpeciesNetClassifier to run. input_queue: Input queue of preprocessed images. results_dict: Output dict for inference results. batch_size: Batch size for inference. csg|]}qSr/)rj)rf_r^r/r0rh:z#_run_classifier..cSg|]}|dqS)rr/rftr/r/r0rh;rvcSrw)r/rxr/r/r0rh<rvN)range batch_predictzip) rnr^r_rs input_tuples filepathsimgs predictionsrXrlr/rur0_run_classifier#s  rcountry admin1_regionlatitude longitudecCs$t||||}||||d||<dS)aFind the first-level administrative division for a given (lat, lon) location. This function uses the provided geographic information to find the first-level administrative division (e.g., state in the USA) using the `find_admin1_region()` function. The result is stored in the `results_dict`. Args: filepath: Image filepath. country: Country in ISO 3166-1 alpha-3 format. Optional. admin1_region: First-level administrative division in ISO 3166-2 format. Optional. latitude: Float value representing latitude. Optional. longitude: Float value representing longitude. Optional. results_dict: Output dict for geolocation results. rrrrNr)rXrrrrr_ admin1_resultr/r/r0_find_admin1_regionBs rensemblerclassifier_resultsdetector_resultsgeolocation_resultspartial_predictionspredictions_json save_lockc Csj|j|||||d}d|i} |r3|r,|t| |WddS1s%wYdSt| |dS| S)aCombines inference results from multiple jobs that ran independently. Brings together results from the classifier, detector, and geolocation steps to create the final predictions. The SpeciesNet ensemble model is used to combine these results, which may be saved to a JSON file. Args: ensemble: SpeciesNetEnsemble to run. filepaths: List of filepaths to ensemble predictions for. classifier_results: Dict of classifier results, with keys given by the filepaths to ensemble predictions for. detector_results: Dict of detector results, with keys given by the filepaths to ensemble predictions for. geolocation_results: Dict of geolocation results, with keys given by the filepaths to ensemble predictions for. partial_predictions: Dict of partial predictions from previous ensemblings, with keys given by the filepaths for which predictions where already ensembled. Used to skip re-ensembling for the matching filepaths. predictions_json: Output filepath where to save the predictions dict in JSON format. If `None`, predictions are not saved to a file and are returned instead. save_lock: Threading lock used to avoid race conditions when saving predictions to a file. Only needed when `predictions_json` is not `None`, otherwise it is ignored. Returns: The predictions dict of ensembled inference results if `predictions_json` is set to `None`, otherwise return `None` since predictions are saved to a file. )rrrrrrN)combiner) rrrrrrrrensemble_resultspredictions_dictr/r/r0_combine_resultshs/ "rnew_predictions failure_typec Csg}|D]$}||vr|||q||vr|||q|||jgdqd|i}|rR|rK|t||WddS1sDwYdSt||dS|S)a Merges new inference results with partial predictions from previous runs. Args: filepaths: List of filepaths to merge predictions for. new_predictions: Dict of new inference results keyed by filepaths. partial_predictions: Dict of partial predictions from previous runs, keyed by filepaths. failure_type: Type of failure to report when a prediction is missing. predictions_json: Output filepath where to save the predictions dict in JSON format. If `None`, predictions are not saved to a file and are returned instead. save_lock: Threading lock used to avoid race conditions when saving predictions to a file. Only needed when `predictions_json` is not `None`, otherwise it is ignored. Returns: The predictions dict of merged inference results if `predictions_json` is set to `None`, otherwise return `None` since predictions are saved to a file. )rXfailuresrN)appendrNr) rrrrrrresultsrXrr/r/r0_merge_resultss(  "rrcOs.t}t|d||d|}|||fS)aZStarts periodic results saving every 10 minutes. Initiates a background thread to save partial results periodically to avoid losing progress during longer inference jobs. Args: fn: Callable for saving results. *fn_args: Arguments for the callable that saves results. **fn_kwargs: Keyword arguments for the callable that saves results. Returns: A tuple made of: (a) a repeated action to save results periodically, and (b) a threading lock used to avoid race conditions when saving predictions to a file. iX)rrr)r'Lockrstart)rr+r,rperiodic_saverr/r/r0_start_periodic_results_savingsrrcCs|r ||dSdS)zkStops periodic results saving. Args: periodic_saver: Repeated action to stop. N)r6join)rr/r/r0_stop_periodic_results_saving s rec Cs&tddtt|||jdS)zTError callback to log inference errors. Args: e: Exception to log. zUnexpected inference error: %sN)r errorr tracebackformat_exceptiontype __traceback__)rr/r/r0_error_callbacksrc@s&eZdZdZdddedddeded d ed eed e d eddfddZ dLddZ  dMde dedee dee fddZ        dNde dededee dee dee dee dee dee dee fdd Z   dOde dededee dee f d!d"Z   dOde dededee dee f d#d$Z        dPde d%ee dededee dee dee dee dee dee fd&d'Z    dQde d%ee dededee dee f d(d)Z    dQde d%ee dededee dee f d*d+Z      dRde dedee dee dee dee dee dee fd,d-Z  dMde dedee dee fd.d/Z  dMde dedee dee fd0d1Z    dSde d2ee d%ee dedee dee f d3d4Zddddddddd5dddd6 dee d7ee d8eee d9ee d:eee d;ee ded?dededee dee fd@dAZdddddddddd5ddddB dee d7ee d8eee d9ee d:eee d;ee dedCdededee dee fdDdEZddddddddd5dddF dee d7ee d8eee d9ee d:eee d;ee dedCdedee dee fdGdHZdddddddddddddI dee d7ee d8eee d9ee d:eee d;ee d.rrrrrrrNrCrGrHrIrJr?r@rBrXrrrrrCrGracSrbrcrrer/r/r0rhrirprHrIrrJrrrrrrrr)rlenrrrr>r'rVrjrrWrZrOrkrnrr6r)r*rrrrrrrrrinstances_to_processnum_instances_to_processrrprogressrrXrrrrr\detector_inputrarqclassifier_inputrr/r/r0_predict_using_single_threadrs             z'SpeciesNet._predict_using_single_threadrs new_pool_fn new_list_fn new_dict_fn new_queue_fn new_rlock_fnc  s|dusJ|dus J|dusJ|dusJ| dusJ|d} |dd| D} |} |} |}t|| \}}||}t|}||t||d}||}|sV|}|rhtt|j| | | |||d\}}nd}d}t|rsgdng||| d|}|d}|d}|td |d }|}|td |d }|D]G}|jt |j |d |ffd d t d|jt |j ||ffdd t d|jt|d |d|d|d|d|ffdd t dqt|D]}|jt|j || |ffdd t dqt|D]}|jt|j || ||dkr|n|ffdd t dq|||||||rDt|t|j| | | ||||dS)ahRuns prediction using worker pools (multi-threading or multiprocessing). This method uses worker pools to process images concurrently. This is an abstract method that accepts worker pools of different types. Args: instances_dict: Instances dict to process. batch_size: Batch size for inference. progress_bars: Whether to show progress bars. predictions_json: Path where to save the JSON output. new_pool_fn: Callable that returns a new pool. new_list_fn: Callable that returns a list. new_dict_fn: Callable that returns a dict. new_queue_fn: Callable that returns a queue. new_rlock_fn: Callable that returns a thread/process lock. Returns: The predictions dict of ensembled inference results if `predictions_json` is set to `None`, otherwise return `None` since predictions are saved to a file. NrcSrwrr/rr/r/r0rh+rvz:SpeciesNet._predict_using_worker_pools..rzrrr?r@rArB@rXc dSNrCrOrtrr/r0w z8SpeciesNet._predict_using_worker_pools..argscallbackerror_callbackcrNrHrrrr/r0rrrrrrcr)NrJrrrr/r0rrcrNrGrrrr/r0rrcrNrIrrrr/r0rrr)rrminrrrr>max apply_asyncr]rWrrrrnrrjr{rmrrQrr6r)r*rrsrrrrrrrrrrrrrrr num_batcheslast_batch_sizerr common_pool detector_poolclassifier_poolrYr`rorrt batch_idxr/rr0_predict_using_worker_poolss *                    z&SpeciesNet._predict_using_worker_poolsc Cs |j||||ttttjtjd SN)rsrrrrrrr)rrrSdictqueueQueuer'rVr*rrsrrr/r/r0_predict_using_thread_poolssz&SpeciesNet._predict_using_thread_poolsc Cs<|jdusJ|j||||tj|jj|jj|jj|jjd Sr)rrmpPoolrSrrrVrr/r/r0_predict_using_process_poolssz'SpeciesNet._predict_using_process_poolsdetections_dictc  s|dusJ|dus J|dusJ| dusJ|d} dd| D} |p&i}|} t|| \} }|| } t|}||t||d}||}|sL|}|r\tt| | | tj|d\}}nd}d}t|rgddgng||| d|}|d}|}|td |d }|D]-}|d }| |i d g}| |d d|Df|j t |j ||ffddtdqt|D]}|j t|j || ||dkr|n|ffddtdq|||||rt|t| | | tj||dS)NrcSrwrr/rr/r/r0rhrvz;SpeciesNet._classify_using_worker_pools..rzrrrrrrHrIrrrrXracSrbrcrrer/r/r0rh7ricrrrrrr/r0r;rz9SpeciesNet._classify_using_worker_pools..rcrrrrrr/r0rIrrrrrrr)rrrrrr CLASSIFIERr>rrjr[rrrrnrr{rrQrr6r)r*rrrsrrrrrrrrrrrrrrrrrrr`rorrXrarr/rr0_classify_using_worker_poolss            z'SpeciesNet._classify_using_worker_poolsc Cs |j|||||tttjtjd SN)rrsrrrrrr)rrrrrr'rVr*rrrsrrr/r/r0_classify_using_thread_poolsdsz'SpeciesNet._classify_using_thread_poolsc Cs8|jdusJ|j|||||tj|jj|jj|jjd Sr)rrrrrrrVrr/r/r0_classify_using_process_poolsxsz(SpeciesNet._classify_using_process_poolscsn|dusJ|dus J|dusJ|dusJ|d}dd|D} |} t||\} } || } t| } |rEtt| | | tj|d\}}nd}d}t|rPddgng| |d|}|d}|d }| D]}|jt|j |d |ffd d t d qdt | D]}|jt |j || ffdd t d q~| | |||rt|t| | | tj||dS)NrcSrwrr/rr/r/r0rhrvz9SpeciesNet._detect_using_worker_pools..rrCrGrrzrrXcrrrrrr/r0rrz7SpeciesNet._detect_using_worker_pools..rcrrrrrr/r0rrr)rrrrr DETECTORr>rr]rWrr{rmrQrr6r)r*rrrrrrrrrrrrrrrrrrYrrtr/rr0_detect_using_worker_poolss~          z%SpeciesNet._detect_using_worker_poolsc Cs|j|||tttjtjdSN)rrrrrr)rrrrrr'rVr*rrrr/r/r0_detect_using_thread_poolssz%SpeciesNet._detect_using_thread_poolsc Cs4|jdusJ|j|||tj|jj|jj|jjdSr)rrrrrrrVrr/r/r0_detect_using_process_poolssz&SpeciesNet._detect_using_process_poolsclassifications_dictc Cs|d}dd|D}|pi}|pi} i} t||\} } t| } |r2tt|j||| | | |d\}}nd}d}t|r.rrJrrXrrrrrr) rrrrrr>r'rVrjrrOr6r)r*rrrrrrrrrrrrrrrrrrXrrrrrr/r/r0_ensemble_using_single_threadsn        z(SpeciesNet._ensemble_using_single_thread multi_thread) rinstances_jsonr filepaths_txtfolders folders_txtrrrun_modersrrrrrrrrrr) single_threadr multi_processc Csrt||||||||}| dkr|j|| | dS| dkr$|j|| | | dS| dkr1|j|| | | dStd| d)Nrrrr)rsrrrUnknown run mode: ``)rrrr ValueError) r*rrrrrrrrrrsrrr/r/r0rkes> zSpeciesNet.predict) rrrrrrrrrrrsrr)rrc  Cs^t||||||||}| dkr|j|| | | | dS| dkr'|j|| | | | dStd| d)Nr)rrsrrrr r )rrrr )r*rrrrrrrrrrrsrrr/r/r0classifys6 zSpeciesNet.classify) rrrrrrrrrrrc CsVt||||||||}| dkr|j|| | dS| dkr#|j|| | dStd| d)Nrr rr r )rrrr ) r*rrrrrrrrrrrr/r/r0detects. zSpeciesNet.detect) rrrrrrrrrrrrc Cs*t||||||||}|j|| | | | dS)N)rrrr)rr) r*rrrrrrrrrrrrr/r/r0ensemble_from_past_runss" z"SpeciesNet.ensemble_from_past_runsr7)FN)rFNNNNNN)rFN)NrFNNNNN)NrFN)FNNNNN)NNFN)r8r9r:r;rrTrboolrrr"rrStrPathrrUrrrrrrrrrrrSrkr rrr/r/r/r0r#s  =      N     |   g   V      6        1        +       )N)NN)Dr;__all__rrmultiprocessing.managersrmultiprocessing.poolrpathlibrrr'rtypingrrrr abslr r speciesnet.classifierr speciesnet.constantsr speciesnet.detectorrspeciesnet.ensembler'speciesnet.ensemble_prediction_combinerrspeciesnet.geolocationrspeciesnet.utilsrrrrrrrTrtuple DetectorInputrS BBoxOutputClassifierInputregisterThreadrr>rr]rrmrrrUrr<rrrrrr Exceptionrrr/r/r/r0s                      -\      -     F   =  #