o Ďii@s@ddlmZddlZddlZddlZddlZddlZddlmZm Z ddl m Z ddl m Z ddlmZddlmZmZddlmZddlZdd lmZdd lmZdd lmZmZmZdd lmZmZm Z Gd ddZ!GdddZ"Gddde"Z#Gddde"Z$GdddZ%Gddde%Z&GdddZ'GdddZ( dgddZ) dhd d!Z*did&d'Z+djd,d-Z,dkdld1d2Z-dmd4d5Z.dnd8d9Z/dod;d<Z0dpd>d?Z1dpd@dAZ2 B B C CdqdrdIdJZ3e GdKdLdLZ4dMdNZ5GdOdPdPZ6dkdsdUdVZ7   W dtdud^d_Z8   W dtdud`daZ9 dvdwdcddZ: dvdwdedfZ;dS)x) annotationsN)Counter defaultdict) dataclass)Pool)Path)LockThread)Literal)tqdm)logger)is_colab load_json save_json)ShapelyAnnotationboxget_shapely_multipolygonc@s<eZdZdZddd d Zed d Zed dZddZ dS) CocoCategoryzCOCO formatted category.rNidintname str | None supercategorycCs(t||_||_|r||_dS||_dSN)rrrr)selfrrrrK/home/jeff/fluffinator/venv/lib/python3.10/site-packages/sahi/utils/coco.py__init__s zCocoCategory.__init__cCs.||d|dd|vr|ddS|ddS)zCreates CocoCategory object using coco category. Args: category: Dict {"supercategory": "person", "id": 1, "name": "person"}, rrrrrrr)clscategoryrrrfrom_coco_category szCocoCategory.from_coco_categorycCs|j|j|jdS)Nrrrrrrjson.szCocoCategory.jsoncCsd|jd|jd|jdS)NzCocoCategory< id: , name: z, supercategory: >rr"rrr__repr__6szCocoCategory.__repr__)rNN)rrrrrr) __name__ __module__ __qualname____doc__r classmethodr!propertyr#r&rrrrrs    rc@seZdZdZed5ddZed5ddZed6d7d d Zed8ddZ     d9d:ddZ d;ddZ e ddZ e d d!Z e d"d#Ze d$d%Zejd&d%Ze d'd(Zejd)d(Ze d*d+Zejd,d+Ze d-d.Ze d/d0Zd1d2Zd3d4ZdS)<CocoAnnotationzCOCO formatted annotation.rcC|||||dS)arCreates CocoAnnotation object using coco segmentation. Args: segmentation: List[List] [[1, 1, 325, 125, 250, 200, 5, 200]] category_id: int Category id of the annotation category_name: str Category name of the annotation iscrowd: int 0 or 1 ) segmentation category_id category_nameiscrowdr)rr/r0r1r2rrrfrom_coco_segmentation@ z%CocoAnnotation.from_coco_segmentationcCr.)aSCreates CocoAnnotation object using coco bbox. Args: bbox: List [xmin, ymin, width, height] category_id: int Category id of the annotation category_name: str Category name of the annotation iscrowd: int 0 or 1 bboxr0r1r2r)rr6r0r1r2rrrfrom_coco_bboxUr4zCocoAnnotation.from_coco_bboxNannotation_dictdictr1rcCsz|drt|dtrd}td|ddnd}|dr2|dr2|s2||d|d|dS||d |d|d S) a|Creates CocoAnnotation object from category name and COCO formatted annotation dict (with fields "bbox", "segmentation", "category_id"). Args: category_name: str Category name of the annotation annotation_dict: dict COCO formatted annotation dict (with fields "bbox", "segmentation", "category_id") r/TzSegmentation annotation for id rz; is skipped since RLE segmentation format is not supported.Fr0)r/r0r1r6)r6r0r1) __contains__ isinstancer9r warning)rr8r1has_rle_segmentationrrrfrom_coco_annotation_dictjs, z(CocoAnnotation.from_coco_annotation_dictshapely_annotationrr0rstrr2cCs(|gd|||d}||_||_|S)a'Creates CocoAnnotation object from ShapelyAnnotation object. Args: shapely_annotation (ShapelyAnnotation) category_id (int): Category id of the annotation category_name (str): Category name of the annotation iscrowd (int): 0 or 1 )rrrrr5)to_coco_segmentation _segmentation_shapely_annotation)rr?r0r1r2coco_annotationrrrfrom_shapely_annotations z&CocoAnnotation.from_shapely_annotationr6list[int] | NonecCsn|dur |dur td||_||_||_||_||_|jr&tj|jd}n |s,tdtj |d}||_ dS)aCreates coco annotation object using bbox or segmentation. Args: segmentation: List[List] [[1, 1, 325, 125, 250, 200, 5, 200]] bbox: List [xmin, ymin, width, height] category_id: int Category id of the annotation category_name: str Category name of the annotation image_id: int Image ID of the annotation iscrowd: int 0 or 1 Nz"you must provide a bbox or polygon)r/zCoco bounding box not set)r6) ValueErrorrB _category_id_category_name _image_id_iscrowdrr3 TypeErrorr7rC)rr0r1r/r6image_idr2r?rrrrs  zCocoAnnotation.__init__ slice_bbox list[int]cCsFt|d|d|d|d}|j|}tj||j|jpd|jdS)Nr)r0r1r2)rrCget_intersectionr-rEr0r1r2)rrNshapely_polygonintersection_shapely_annotationrrrget_sliced_coco_annotations z)CocoAnnotation.get_sliced_coco_annotationcCs|jjS)zDReturns area of annotation polygon (or bbox if no polygon available))rCarear"rrrrXszCocoAnnotation.areacCs |jS)zLReturns coco formatted bbox of the annotation as [xmin, ymin, width, height])rCto_xywhr"rrrr6s zCocoAnnotation.bboxcCs|jr|jSgS)z]Returns coco formatted segmentation of the annotation as [[1, 1, 325, 125, 250, 200, 5, 200]])rBrCrAr"rrrr/s zCocoAnnotation.segmentationcC|jS)z-Returns category id of the annotation as int.)rHr"rrrr0zCocoAnnotation.category_idcCt|ts td||_dS)Nzcategory_id must be an integer)r;r ExceptionrHrirrrr0  cCrZ)z*Returns image id of the annotation as int.)rJr"rrrrMr[zCocoAnnotation.image_idcCr\)Nzimage_id must be an integer)r;rr]rJr^rrrrMr`cCrZ)z/Returns category name of the annotation as str.)rIr"rrrr1r[zCocoAnnotation.category_namecCr\)Nzcategory_name must be a string)r;r@r]rI)rnrrrr1 r`cCrZ)z'Returns iscrowd info of the annotation.)rKr"rrrr2r[zCocoAnnotation.iscrowdcC|j|j|j|j|j|jdS)NrMr6r0r/r2rXrcr"rrrr#zCocoAnnotation.jsoncCstjdtdd|jSNz-Use json property instead of serialize methodrQ) stacklevel)warningswarnDeprecationWarningr#r"rrr serialize szCocoAnnotation.serializecCs>d|jd|jd|jd|jd|jd|jd|jdS) NzCocoAnnotation< image_id: , bbox: , segmentation: , category_id: , category_name: , iscrowd: , area: r%)rMr6r/r0r1r2rXr"rrrr&$szCocoAnnotation.__repr__)rr)r8r9r1r)r?rr0rr1r@r2r)NNNNr)r0rr1rr6rF)rNrO)r'r(r)r*r+r3r7r>rErrWr,rXr6r/r0setterrMr1r2r#rjr&rrrrr-=sR  $  *            r-csxeZdZdZedddZedddZeddd Z      ddfdd Ze ddZ ddZ ddZ Z S)CocoPredictionz.Class for handling predictions in coco format.rNcC|||||||dS)aCreates CocoAnnotation object using coco segmentation. Args: segmentation: List[List] [[1, 1, 325, 125, 250, 200, 5, 200]] category_id: int Category id of the annotation category_name: str Category name of the annotation score: float Prediction score between 0 and 1 iscrowd: int 0 or 1 )r/r0r1scorer2rMr)rr/r0r1rtr2rMrrrr32z%CocoPrediction.from_coco_segmentationcCrs)aCreates CocoAnnotation object using coco bbox. Args: bbox: List [xmin, ymin, width, height] category_id: int Category id of the annotation category_name: str Category name of the annotation score: float Prediction score between 0 and 1 iscrowd: int 0 or 1 )r6r0r1rtr2rMr)rr6r0r1rtr2rMrrrr7KruzCocoPrediction.from_coco_bboxcCs:|dr||d|d|||dS||d|d||dS)aCreates CocoAnnotation object from category name and COCO formatted annotation dict (with fields "bbox", "segmentation", "category_id"). Args: category_name: str Category name of the annotation annotation_dict: dict COCO formatted annotation dict (with fields "bbox", "segmentation", "category_id") score: float Prediction score between 0 and 1 r/r0)r/r0r1rtrMr6r6r0r1rMr)rr1r8rtrMrrrr>ds z(CocoPrediction.from_coco_annotation_dictrSr0rr1r@cs"||_tj||||||ddS)a Args: segmentation: List[List] [[1, 1, 325, 125, 250, 200, 5, 200]] bbox: List [xmin, ymin, width, height] category_id: int Category id of the annotation category_name: str Category name of the annotation image_id: int Image ID of the annotation score: float Prediction score between 0 and 1 iscrowd: int 0 or 1 )r/r6r0r1rMr2N)rtsuperr)rr/r6r0r1rMrtr2 __class__rrrs zCocoPrediction.__init__c Cs&|j|j|j|j|j|j|j|jdS)NrMr6rtr0r1r/r2rXrzr"rrrr#szCocoPrediction.jsoncCstjdtdddSre)rgrhrir"rrrrjszCocoPrediction.serializecCsFd|jd|jd|jd|jd|jd|jd|jd|jd S) NzCocoPrediction< image_id: rkrlz , score: rmrnrorpr%)rMr6r/rtr0r1r2rXr"rrrr&s"zCocoPrediction.__repr__)rNr)NNrrSNNr)r0rr1r@)r'r(r)r*r+r3r7r>rr,r#rjr& __classcell__rrrxrrr/s(   &  rrcs@eZdZdZ    ddfd d Zed d ZddZZS)CocoVidAnnotationzCOCOVid formatted annotation. https://github.com/open-mmlab/mmtracking/blob/master/docs/tutorials/customize_dataset.md#the-cocovid-annotation-file Nrr0rr1r@r6rOcs&tj|||||d||_||_dS)a Args: bbox: List [xmin, ymin, width, height] category_id: int Category id of the annotation category_name: str Category name of the annotation image_id: int Image ID of the annotation instance_id: int Used for tracking iscrowd: int 0 or 1 id: int Annotation id )r6r0r1rMr2N)rwr instance_idr)rr0r1r6rMr}r2rrxrrrs zCocoVidAnnotation.__init__c Cs*|j|j|j|j|j|j|j|j|jd S)N rrMr6r/r0r1r}r2rXr~r"rrrr#szCocoVidAnnotation.jsoncCsNd|jd|jd|jd|jd|jd|jd|jd|jd |jd S) NzCocoAnnotation< id: z, image_id: rkrlrmrnz, instance_id: rorpr%r~r"rrrr&s& zCocoVidAnnotation.__repr__)NNrN)r0rr1r@r6rO) r'r(r)r*rr,r#r&r{rrrxrr|s %  r|c@sHeZdZeddZddd d Zd dZddZeddZ ddZ dS) CocoImagecCs ||d|d|d|ddS)a Creates CocoImage object from COCO formatted image dict (with fields "id", "file_name", "height" and "weight"). Args: image_dict: dict COCO formatted image dict (with fields "id", "file_name", "height" and "weight") r file_nameheightwidthrrrrr)r image_dictrrrfrom_coco_image_dict s  zCocoImage.from_coco_image_dictNrr@rrrr int | NonecCs<|rt|n||_||_t||_t||_g|_g|_dS)aCreates CocoImage object. Args: id : int Image id file_name : str Image path height : int Image height in pixels width : int Image width in pixels N)rrrrrr predictions)rrrrrrrrrs     zCocoImage.__init__cC"t|ts td|j|dS)zYAdds annotation to this CocoImage instance. annotation : CocoAnnotation z,annotation must be a CocoAnnotation instanceN)r;r-rLrappendr annotationrrradd_annotation. zCocoImage.add_annotationcCr)zYAdds prediction to this CocoImage instance. prediction : CocoPrediction z,prediction must be a CocoPrediction instanceN)r;rrrLrr)r predictionrrradd_prediction8rzCocoImage.add_predictioncCs|j|j|j|jdS)Nrrr"rrrr#Bs zCocoImage.jsonc Cs&d|jd|jd|jd|jd S)NzCocoImage< id: z, file_name: , height: , width: zO, annotations: List[CocoAnnotation], predictions: List[CocoPrediction]>rr"rrrr&KszCocoImage.__repr__r)rr@rrrrrr) r'r(r)r+rrrrr,r#r&rrrrr s     rcsReZdZdZ   d fdd ZedddZddZed d Z d d Z Z S) CocoVidImagezCOCOVid formatted image. https://github.com/open-mmlab/mmtracking/blob/master/docs/tutorials/customize_dataset.md#the-cocovid-annotation-file Ncs$tj||||d||_||_dS)aCreates CocoVidImage object. Args: id: int Image id file_name: str Image path height: int Image height in pixels width: int Image width in pixels frame_id: int 0-indexed frame id video_id: int Video id rrrrN)rwrframe_idvideo_id)rrrrrrrrxrrr[s zCocoVidImage.__init__cCs||j|j|j|j||dS)zCreates CocoVidImage object using CocoImage object. Args: coco_image: CocoImage frame_id: int 0-indexed frame id video_id: int Video id rrrrrrr)r coco_imagerrrrrfrom_coco_imagexs zCocoVidImage.from_coco_imagecCr)zc Adds annotation to this CocoImage instance annotation : CocoVidAnnotation z/annotation must be a CocoVidAnnotation instanceN)r;r|rLrrrrrrrrzCocoVidImage.add_annotationcCrb)Nrrr"rrrr#rdzCocoVidImage.jsonc Cs6d|jd|jd|jd|jd|jd|jd S)NzCocoVidImage< file_name: rrz , id: z, video_id: z, frame_id: z+, annotations: List[CocoVidAnnotation]>rr"rrrr&szCocoVidImage.__repr__)NNNNN) r'r(r)r*rr+rrr,r#r&r{rrrxrrUs    rc@sHeZdZdZ    ddd d Zd dZddZeddZddZ dS) CocoVideozCOCO formatted video. https://github.com/open-mmlab/mmtracking/blob/master/docs/tutorials/customize_dataset.md#the-cocovid-annotation-file Nrr@rrfps float | NonerrcCs(||_||_||_||_||_g|_dS)a<Creates CocoVideo object. Args: name: str Video name id: int Video id fps: float Video fps height: int Video height in pixels width: int Video width in pixels N)rrrrrimages)rrrrrrrrrrs  zCocoVideo.__init__cCs(t|ts td|jt|dS)zb Adds image to this CocoVideo instance Args: image: CocoImage z"image must be a CocoImage instanceN)r;rrLrrrrrimagerrr add_images zCocoVideo.add_imagecCr)zs Adds CocoVidImage to this CocoVideo instance Args: cocovidimage: CocoVidImage z,cocovidimage must be a CocoVidImage instanceN)r;rrLrr)r cocovidimagerrradd_cocovidimage zCocoVideo.add_cocovidimagecCs|j|j|j|j|jdS)Nrrrrrrr"rrrr#s zCocoVideo.jsonc Cs.d|jd|jd|jd|jd|jd S)NzCocoVideo< id: r$z , fps: rrz!, images: List[CocoVidImage]>)rrrrrr"rrrr&szCocoVideo.__repr__)NNNN) rr@rrrrrrrr) r'r(r)r*rrrr,r#r&rrrrrs    rc@seZdZ      dQdRddZddZddZddZdSdTddZdUddZe      dVdWd%d&Z e d'd(Z e d)d*Z e d+d,Ze d-d.Ze d/d0Zd1d2ZdXd5d6Z 7 4  dYdZd?d@Z 7 4  dYdZdAdBZd[d\dGdHZd[d]dJdKZd4edLdfdMdNZdOdPZdS)^CocoNFautorr image_dirremapping_dictdict[int, int] | Noneignore_negative_samplesboolclip_bboxes_to_img_dimsimage_id_settingLiteral['auto', 'manual']cCsJ|dvrtd||_||_||_||_g|_g|_d|_||_||_ dS)aCreates Coco object. Args: name: str Name of the Coco dataset, it determines exported json name. image_dir: str Base file directory that contains dataset images. Required for dataset merging. remapping_dict: dict {1:0, 2:1} maps category id 1 to 0 and category id 2 to 1 ignore_negative_samples: bool If True ignores images without annotations in all operations. image_id_setting: str how to assign image ids while exporting can be auto -> will assign id from scratch (.id will be ignored) manual -> you will need to provide image ids in instances (.id can not be None) rmanualz2image_id_setting must be either 'auto' or 'manual'N) rGrrrr categoriesr_statsrr)rrrrrrrrrrrs z Coco.__init__cCV|D]&}|jdur |jD]}|d|kr|j|}||d<q|t|qdSa5Creates CocoCategory object using coco category list. Args: coco_category_list: List[Dict] [ {"supercategory": "person", "id": 1, "name": "person"}, {"supercategory": "vehicle", "id": 2, "name": "bicycle"} ] Nrrkeys add_categoryrr!rcoco_category_list coco_category source_id target_idrrr&add_categories_from_coco_category_list   z+Coco.add_categories_from_coco_category_listcCr)z_Adds category to this Coco instance. Args: category: CocoCategory (category must be a CocoCategory instanceNr;rrLrrrr rrrr3s zCoco.add_categorycCs,|jdkr|jdurtd|j|dS)zVAdds image to this Coco instance. Args: image: CocoImage rNz=image id should be manually set for image_id_setting='manual')rrrGrrrrrrr?szCoco.add_imagedesired_name2iddict[str, int]update_image_filenamescCsJi}t|j|j|j|jd}|jD]!}|j}|j}|s!tdq|| vr.||||<qd||<q| D]}t ||||d} | | q7t |jD]P} t| j} tj| j| jkrbdnd} |r| s|jsqtdntttj|j| j| _| jD]} | j}||}|dur|| _| | q|| qN|j|_dS)axRearranges category mapping of given COCO object based on given desired_name2id. Can also be used to filter some of the categories. Args: desired_name2id: dict {"big_vehicle": 1, "car": 2, "human": 3} update_image_filenames: bool If True, updates coco image file_names with absolute file paths. rrrrz.no category name provided to update categoriesNrTFzimage directory not set)rrrrrrrr r<rrrcopydeepcopyrrrr#ospathabspathrerrorr@rrr0rr__dict__)rrrcurrentid2desiredid_mapping updated_cocorcurrent_category_idcurrent_category_namerupdated_coco_categoryrupdated_coco_imagefile_name_is_abspathrDdesired_category_idrrrupdate_categoriesJsF             zCoco.update_categoriesrPc Cs|jdus |jdurtd|r|std|}|}d}|durEi}||fD]}t|j}|D]}|d|vrC|||d<|d7}q0q0q&||fD] }|j|ddqI|j|j|j|_|j |_ |rltd |jdSdS) agCombines the images/annotations/categories of given coco object with current one. Args: coco : sahi.utils.coco.Coco instance A COCO dataset object desired_name2id : dict {"human": 1, "car": 2, "big_vehicle": 3} verbose: bool If True, merging info is printed Nz)image_dir should be provided for merging.='desired_name2id' is not specified, combining all categories.rrrPT)rrCategories are formed as: ) rrGprintrrjson_categoriesrrextendr) rcocorverbosecoco1coco2 category_indtemp_categories temp_categoryrrrmerges:       z Coco.merge coco_dict_or_path dict | str dict | None use_threads num_threadsrc s|||||d}t|ttfvrtdt|trt|} n|} t| d} || dt| } |j t } t } fdd}| |}|durt |D]#}||}||}|| kr\| }t |||| d| | || fd}|qLt}tD] }||ur|qxnXt| dd D]P}t|}|d }|| vrtd |d q| || |}|D]'}|jd ur|j|d}||d<n|d}|}tj||d}||q||q|r|}|S)a>Creates coco object from COCO formatted dict or COCO dataset file path. Args: coco_dict_or_path: dict/str or List[dict/str] COCO formatted dict or COCO dataset file path List of COCO formatted dict or COCO dataset file path image_dir: str Base file directory that contains dataset images. Required for merging and yolov5 conversion. remapping_dict: dict {1:0, 2:1} maps category id 1 to 0 and category id 2 to 1 ignore_negative_samples: bool If True ignores images without annotations in all operations. clip_bboxes_to_img_dims: bool = False Limits bounding boxes to image dimensions. use_threads: bool = False Use threads when processing the json image list, defaults to False num_threads: int = 10 Slice the image list to given number of chunks, defaults to 10 Properties: images: list of CocoImage category_mapping: dict )rrrrz)coco_dict_or_path should be a dict or strrrcst|||d|d|D]X}t|}|d} | |vr'td| dq||| ||| } | D]'} |jdurM|j| d} | | d<n| d} | } tj | | d}| |q:| |qdS)Nz!Loading coco annotations between z and rduplicate image_id: , will be ignored.r0r1r8) r rrracquireaddreleaserr-r>rr)startfinish image_list _image_id_set_image_id_to_annotation_list_cocolockcoco_image_dictrrMannotation_listcoco_annotation_dictr0r1rDcategory_mappingrrfill_image_id_sets0      z6Coco.from_coco_dict_or_path..fill_image_id_setT)targetargszLoading coco annotationsrrrNr0r)typer@r9rLr;rlenr"get_imageid2annotationlist_mappingrsetrranger r threading currentThread enumeratejoinr rrrrrr-r>rrget_coco_with_clipped_bboxes)rrrrrrrrr coco_dict dict_sizeimage_id_to_annotation_list image_id_setrr chunk_sizer_rrt main_threadrrrMrrr0r1rDrrrfrom_coco_dict_or_pathst#    "         zCoco.from_coco_dict_or_pathcC g}|jD]}||jq|Srrrr#rrr rrrrO zCoco.json_categoriescC i}|jD]}|j||j<q|Srrrrrrr rrrrVrzCoco.category_mappingcCst|j|j|j|jdS)N)rrrr)create_coco_dictrrrrr"rrrr#]s z Coco.jsoncCst|j|j|jdS)N)rrr)create_coco_prediction_arrayrrrr"rrrprediction_arrayfs zCoco.prediction_arraycCs|js||jSr)rcalculate_statsr"rrrstatsnsz Coco.statscCsd}t|j}d}t|j}dd|jD}dd|jD}t|}t|}t|} t|} td} d} d} d}d}|jD]p}i}|jD];}|j}| |7} ||jd7<d||j<||kre|}||krk|}|| |jkrw|| |j<|| |jkr|| |j<qHt|jdkr|d7}|t|j7}t t |t |}t|j}|| kr|} || kr|} qA||dkr|||}| |}nd}d}||||||| | ||||| | d|_ d S) z=Iterates over all annotations and calculates total number of.rcSsi|]}|ddqS)rrr.0r rrr {z(Coco.calculate_stats..cSsi|] }|dtdqS)rinf)floatrrrrr|r g _BrP) num_imagesnum_annotationsnum_categoriesnum_negative_imagesnum_images_per_categorynum_annotations_per_categorymin_num_annotations_in_imagemax_num_annotations_in_imageavg_num_annotations_in_imagemin_annotation_areamax_annotation_areaavg_annotation_area min_annotation_area_per_category max_annotation_area_per_categoryN) rrrrrr!rrXr1r9rr)rr$r#r&r%category_name_to_zerocategory_name_to_infr'r(r/r0r)r*total_annotation_arear,r-rimage_contains_categoryrannotation_areanum_annotations_in_imager+r.rrrrtsz                zCoco.calculate_stats?rc Cst|j}t|j}t|t|t||}|d|}||d}t|j r.|j nd|j d}||_|j |_ t|j rB|j nd|j d} || _|j | _ || dS)aSplit images into train-val and returns them as sahi.utils.coco.Coco objects. Args: train_split_rate: float numpy_seed: int random seed. Actually, this doesn't use numpy, but the random package from the standard library, but it is called numpy for compatibility. Returns: result : dict { "train_coco": "", "val_coco": "", } N split_train)rr split_val) train_cocoval_coco) rrrrrandomseedshufflerrrrr) rtrain_split_rate numpy_seedr#shuffled_images num_train train_images val_imagesr:r;rrrsplit_coco_as_train_vals&       zCoco.split_coco_as_train_val? output_dir str | Pathr?r!r@mpdisable_symlinkcCs$tdt|j|||||ddS)znDeprecated. Please use export_as_yolo instead. Calls export_as_yolo with the same arguments. zBexport_as_yolov5 is deprecated. Please use export_as_yolo instead.)rGr?r@rIrJN)rgrhriexport_as_yolo)rrGr?r@rIrJrrrexport_as_yolov5s  zCoco.export_as_yolov5cCszddl}Wn tytdwd|kr|dkrd}n|dkr$d}n |dkr+d}ntd|dkrC|j||d }|d } |d } n|dkrL|} d} n|dkrTd} |} d } d } |d vrmttj|d} | jddd|dvrttj|d} | jddd|d vrt | | |j ||d|dvrt | | |j ||dt | t | t |j t|j d} t t|d}t|d}|j| |ddWddS1swYdS)a'Exports current COCO dataset in ultralytics/yolo format. Creates train val folders with image symlinks and txt files and a data yaml file. Args: output_dir: str Export directory. train_split_rate: float If given 1, will be exported as train split. If given 0, will be exported as val split. If in between 0-1, both train/val splits will be calculated and exported. numpy_seed: int To fix the numpy seed. mp: bool If True, multiprocess mode is on. Should be called in 'if __name__ == __main__:' block. disable_symlink: bool If True, symlinks will not be created. Instead, images will be copied. rNzVPlease run "pip install -U pyyaml" to install yaml first for yolo formatted exporting.rPTRAINVALVALTRAIN#train_split_rate cannot be <0 or >1r?r@r:r;rS)rMrOtrain/Tparentsexist_ok)rMrNval/rGrrrIrJtrainvalncnamesdata.ymlwdefault_flow_style)yaml ImportErrorrGrErrrrmkdir,export_yolo_images_and_txts_from_coco_objectrr@rrlistvaluesopendump)rrGr?r@rIrJra split_moderesultr:r; train_dirval_dirdata yaml_pathoutfilerrrrKst      "zCoco.export_as_yolorQsubsample_ratior0rc CsXt|j|j|j|jd}||j|durg}|jD]/}tt }|j D]}d||j <q$||r3d}n|dkrAt |j dkrAd}nd}|rJ| |qg} |jD]/}tt }|j D]}d||j <qY||rhd}n|dkrvt |j dkrvd}nd}|r| |qP|r|} tt | D] } || | qn|j} tdt | |D] } || | q|S)aSubsamples images with subsample_ratio and returns as sahi.utils.coco.Coco object. Args: subsample_ratio: int 10 means take every 10th image with its annotations category_id: int subsample only images containing given category_id, if -1 then subsamples negative samples Returns: subsampled_coco: sahi.utils.coco.Coco rNrPTrF)rrrrrrrrrrrr0rrrr) rrpr0subsampled_cocoimages_that_contain_categoryrcategory_id_to_containsradd_this_image#images_that_doesnt_contain_categoryselected_images image_indrrrget_subsampled_coco]sV          zCoco.get_subsampled_cocoupsample_ratioc Cst|j|j|j|jd}||jt|D]P}tt|j D]F}|durYt t }|j |j D]}d||j <q/||r>d}n|dkrOt|j |j dkrOd}n |dkrVd}nd}nd}|re||j |qq|S)aUpsamples images with upsample_ratio and returns as sahi.utils.coco.Coco object. Args: upsample_ratio: int 10 means copy each sample 10 times category_id: int upsample only images containing given category_id, if -1 then upsamples negative samples Returns: upsampled_coco: sahi.utils.coco.Coco rNrPTrqrF)rrrrrrrrrrrrrr0r) rrzr0upsampled_cocoindrxrtrrurrrget_upsampled_cocos4    zCoco.get_upsampled_cocor c Cst|j|j|j|jd}||j|jD]B}d}|jD]3}|durC|j | vrC||j d}||j d} |j |ksA|j | krCd}|j |ksM|j |krOd}q|rW| |q|S)aFilters annotation areas with given min and max values and returns remaining images as sahi.utils.coco.Coco object. Args: min: int minimum allowed area max_val: int maximum allowed area intervals_per_category: dict of dicts { "human": {"min": 20, "max": 10000}, "vehicle": {"min": 50, "max": 15000}, } Returns: area_filtered_coco: sahi.utils.coco.Coco rTNminmaxF) rrrrrrrrrr1rrXr) rr~max_valintervals_per_categoryarea_filtered_cocoris_valid_imagercategory_based_mincategory_based_maxrrrget_area_filtered_cocos,    zCoco.get_area_filtered_cococ Csddlm}t|j|j|j|jd}||j|j D]F}dd|j |j g}t |j |j |j |jd}|jD]'}|j}|||dr[||}t|j} t| |j|j|jd} || q4q4||q|S)z6Limits overflowing bounding boxes to image dimensions.r)annotation_inside_slicerr)rrNrv) sahi.slicingrrrrrrrrrrrrrrrr#rWrrYrCr-r0r1rMrr) rrrcoco_imgimg_dimsrcoco_annann_dict shapely_annr6coco_ann_from_shapelyrrrrs8         z!Coco.get_coco_with_clipped_bboxes)NNNFFr) rrrrrrrrrrrr)F)rrrrNrP)NNFFFr)rrrrrrrrrrrrrr)r7r)rFrFF) rGrHr?r!r@rrIrrJr)rQN)rprr0r)rzrr0r)r'r(r)rrrrrrr+rr,rrr#rrrrErLrKryr}r!rrrrrrrs` % =0      G.  _ C, 'rFcstdtrstdd|rDtdd"}fddjD}|tt|t |dWd d S1s=wYd StjD] }t|j qId S) aWCreates image symlinks and annotation txts in yolo format from coco dataset. Args: output_dir: str Export directory. coco: sahi.utils.coco.Coco Initialized Coco object that contains images and categories. ignore_negative_samples: bool If True ignores images without annotations in all operations. mp: bool If True, multiprocess mode is on. Should be called in 'if __name__ == __main__:' block. disable_symlink: bool If True, symlinks are not created. Instead images are copied. z:generating image symlinks and annotation files for yolo...z2symlink is not supported in colab, disabling it...T0) processescsg|] }|jfqSrr)rrrrJrrGrr /sz@export_yolo_images_and_txts_from_coco_object..)totalN) r infor r<rrstarmap.export_single_yolo_image_and_corresponding_txtr rr)rGrrrIrJpoolrrrrrrds&    "  rdc Csd}|jD] }t|jdkrd}nq|rdSt|jdkr"|r"dSt|jjdkr5td|jddSt|jjd vrHtd |jddSt|jrWtj |j}n|dur_t d tj t t||j}t t|t|jj }t|} d } t| rt|j} t|j} t|j} | d t | } t | | | } | d7} t| s|rddl}||| nt|| |j}|j}d|}d|}t| j}| |d}|j}t|d^}|D]R}|jd|jd d}|jd|jdd}|jd }|jd}||}||}||}||}|j}||||f}|t |dddd|DdqWddS1sDwYdS)a?Generates YOLO formatted image symlink and annotation txt file. Args: coco_image: sahi.utils.coco.CocoImage coco_image_dir: str output_dir: str Export directory. ignore_negative_samples: bool If True ignores images without annotations in all operations. FTNrrSz(image file has no suffix, skipping it: '').txtz/image file has incorrect suffix, skipping it: 'zAYou have to specify image_dir of Coco object for yolo conversion.rQ_rPrFrr^g@rR cSsg|]}t|qSr)r@)rvaluerrrrzBexport_single_yolo_image_and_corresponding_txt.. )rrr6rrsuffixris_filerrrrGr@rrrparentstemshutilsymlinkrrreplacergr0writer)rcoco_image_dirrGrrJcontains_invalid_annotationsrDcoco_image_pathyolo_image_path_tempyolo_image_pathname_increment parent_dirfilename filesuffixrrrdwdhimage_file_suffixyolo_annotation_pathrrorx_centery_center bbox_width bbox_heightr0 yolo_bboxrrrr>st              ,$rrr9rreturnc Cst|}gggd}i}|dD]}|d}|d}||vr(||||<qd||<q|dD]}|d}||} | dkrJ| |d<|d|q1g} |D]} i}| |d<|d<|| |d<| |qQ| |d<|d |d <|S) aRearranges category mapping of given COCO dictionary based on given category_mapping. Can also be used to filter some of the categories. Arguments: --------- desired_name2id : dict {"big_vehicle": 1, "car": 2, "human": 3} coco_dict : dict COCO formatted dictionary. Returns: --------- coco_target : dict COCO dict with updated/filtered categories. rrrrrrrqrr0rr)rrrr) rr coco_source coco_targetrr rrrrrrrrrrs2          r coco_pathr@ save_pathNonecCs t|}t||}t||dS)aDRearranges category mapping of a COCO dictionary in coco_path based on given category_mapping. Can also be used to filter some of the categories. Arguments: --------- desired_name2id : dict {"human": 1, "car": 2, "big_vehicle": 3} coco_path : str "dirname/coco.json" N)rrr)rrrrrrrrupdate_categories_from_files r coco_dict1 coco_dict2rc Cst|}t|}|durt||}t||}|d|dkr.dd|dD}t||}tdd|dD}tdd|d D}|}|dD]}|d |d 7<|d|qP|d D]} | d |d 7<| d |d 7<|d | qh|S) aCombines 2 coco formatted annotations dicts, and returns the combined coco dict. Arguments: --------- coco_dict1 : dict First coco dictionary. coco_dict2 : dict Second coco dictionary. desired_name2id : dict {"human": 1, "car": 2, "big_vehicle": 3} Returns: --------- merged_coco_dict : dict Merged COCO dict. NrcSsi|] }|d|dqS)rrrrrrrrr"zmerge..cSg|]}|dqSrr)rrrrrrrzmerge..rcSrrr)rrrrrrrrrrPrM)rrrnparrayrr) rrrtemp_coco_dict1temp_coco_dict2 max_image_idmax_annotation_idmerged_coco_dictrrrrrrs&       rrPcCs|r|std|dur2i}d}|D]}t|d}|D]}|d|vr0|||d<|d7}qqqt|D]\}}|dkrDt|}q6t|||}q6|rTtd|d|S)aCombines a list of coco formatted annotations dicts, and returns the combined coco dict. Arguments: --------- coco_dict_list: list of dict A list of coco dicts desired_name2id: dict {"human": 1, "car": 2, "big_vehicle": 3} verbose: bool If True, merging info is printed Returns: --------- merged_coco_dict: dict Merged COCO dict. rNrrrrPr)rrrrr)coco_dict_listrrr|rrrrrrrmerge_from_lists0    r coco_path1 coco_path2cCs(t|}t|}t||}t||dS)aJCombines 2 coco formatted annotations files given their paths, and saves the combined file to save_path. Arguments: --------- coco_path1 : str Path for the first coco file. coco_path2 : str Path for the second coco file. save_path : str "dirname/coco.json" N)rrr)rrrrrrrrrmerge_from_fileHs rdict[int, list[CocoAnnotation]]cCs:tt}td|dD] }|d}|||q |S)aGet image_id to annotationlist mapping for faster indexing. Arguments --------- coco_dict : dict coco dict with fields "images", "annotations", "categories" Returns ------- image_id_to_annotation_list : dict { 1: [CocoAnnotation, CocoAnnotation, CocoAnnotation], 2: [CocoAnnotation] } where CocoAnnotation = { 'area': 2795520, 'bbox': [491.0, 1035.0, 153.0, 182.0], 'category_id': 1, 'id': 1, 'image_id': 1, 'iscrowd': 0, 'segmentation': [[491.0, 1035.0, 644.0, 1035.0, 644.0, 1217.0, 491.0, 1217.0]] } z$indexing coco dataset annotations...rrM)rrer debugr)rr rrMrrrr`s   rrc Cs|dvrtdd}d}tgg|d}|D]Z}|j}t|} |r%| dkr%q|dkr0|} |d7}n|dkr@|jdur=td |j} |j|j| |jd } |d | |D]} d| | j | j | j || j d } |d | |d7}qSq|S)aCreates COCO dict with fields "images", "annotations", "categories". Arguments --------- images : List of CocoImage containing a list of CocoAnnotation categories : List of Dict COCO categories ignore_negative_samples : Bool If True, images without annotations are ignored image_id_setting: str how to assign image ids while exporting can be auto --> will assign id from scratch (.id will be ignored) manual --> you will need to provide image ids in instances (.id can not be None) Returns ------- coco_dict : Dict COCO dict with fields "images", "annotations", "categories" r6'image_id_setting' should be one of ['auto', 'manual']rPrrrrNH'coco_image.id' should be set manually when image_id_setting == 'manual')rrrrr)r2rMr6r/r0rrXr) rGr9rrrrrrrr6r/r0rX)rrrr image_index annotation_idrrcoco_annotationsr$rM out_imagerDout_annotationrrrrsH    rc Cs|dvrtdd}d}g}|D]N}|j}t|}|r |dkr q|dkr+|} |d7}n|dkr;|jdur8td|j} t|D]\} } || | j| j| j| j| j | j d } | | |d7}q?q|S) aCreates COCO prediction array which is list of predictions. Arguments --------- images : List of CocoImage containing a list of CocoAnnotation ignore_negative_samples : Bool If True, images without predictions are ignored image_id_setting: str how to assign image ids while exporting can be auto --> will assign id from scratch (.id will be ignored) manual --> you will need to provide image ids in instances (.id can not be None) Returns ------- coco_prediction_array : List COCO predictions array rrrPrrrNr)rrMr6rtr0r/r2rX) rGrrrrr6rtr0r/r2rXr) rrrr prediction_idpredictions_arrayrcoco_predictionsnum_predictionsrMprediction_indexcoco_predictionout_predictionrrrrs>    rrSTsource_coco_pathtarget_coco_pathadd_bboxradd_areac st|}t|}|d}t|D]c\}}|rdgfdd|dDttdddtdddtdddtdddg\}} } } || | || | f\} } }}| | ||g||d <|rtt|dd }|j||d <q||d<t |||S) zTakes single coco dataset file path, calculates and fills bbox and area fields of the annotations and exports the updated coco dict. Returns: coco_dict : dict Updated coco dict rcsg|]}|qSr)r)r coco_polygon coco_polygonsrrr"rz-add_bbox_and_area_to_coco..r/rNrQrPr6)coco_segmentationrX) rrrrrer~rrrXr)rrrrrrr|rminxminymaxxmaxyxyrrshapely_multipolygonrrradd_bbox_and_area_to_coco s6     rc@s2eZdZUdZded<ded<ddZdd Zd S) DatasetClassCountszDStores the number of images that include each category in a dataset.r9countsr total_imagescsfddjDS)z>Calculates the frequency of images that contain each category.csi|] \}}||jqSr)r)rcidcountr"rrrFsz2DatasetClassCounts.frequencies..)ritemsr"rr"r frequenciesDszDatasetClassCounts.frequenciescCsx|j|j}t|jt|j}i}|jD]\}}||j|d||<q|D] }|j|||<q-t||S)Nr)rrrrrgetr)rorexclusive_keysrkvrrr__add__Hs  zDatasetClassCounts.__add__N)r'r(r)r*__annotations__rrrrrrr=s  rc Cstdd}t|}|dD]}|d}|d}|||d|||<qtt}|D]\}}|D]\}}|dkrE||d||<q5q-t|}t|} t|| S)zReads a coco dataset file and returns an DatasetClassCounts object that stores the number of images that include each category in a dataset Returns: DatasetClassCounts object coco_file_path : str path to coco dataset file cSsttSr)rrrrrr[sz,count_images_with_category..rrMr0rPr)rrrrr9rrr) coco_file_pathimage_id_2_category_2_countrrrMrcategory_2_countimage_category_2_countrrrrrcount_images_with_categorySs    rc@sVeZdZdddZddZddd Zed d Zed d ZdddZ eddZ dS)CocoVidNcCs||_||_g|_g|_dS)zCreates CocoVid object. Args: name: str Name of the CocoVid dataset, it determines exported json name. remapping_dict: dict {1:0, 2:1} maps category id 1 to 0 and category id 2 to 1 N)rrrvideos)rrrrrrrns  zCocoVid.__init__cCrrrrrrrr|rz.CocoVid.add_categories_from_coco_category_listr rcCr)zbAdds category to this CocoVid instance. Args: category: CocoCategory rNrrrrrrrzCocoVid.add_categorycCrrrrrrrrrzCocoVid.json_categoriescCrrrrrrrrrzCocoVid.category_mappingvideorcCr)zYAdds video to this CocoVid instance. Args: video: CocoVideo z"video must be a CocoVideo instanceN)r;rrLr r)rr rrr add_videorzCocoVid.add_videoc Csggg|jd}d}d}d}d}|jD]n}||_|d|jd}t}|jD]K} || _|| _|j| _|d| j| j D]%} | | j | j |7_ || _| j| _ |d| jt |d}q?t |d}t |d}q(t |d}|t|7}q|S)N)r rrrrPr rrr)rr rrr#rrrrrrr}rMrrr) rrrrMrglobal_instance_id coco_videorinstance_id_set cocovid_imagecocovid_annotationrrrr#s>    z CocoVid.jsonr)r r)r r) r'r(r)rrrr,rrr r#rrrrrms      rresult_list_or_path list | strdataset_dict_or_pathdict | str | Nonec Cspt|tr t|}n t|tr|}ntd|durKt|tr$t|}n t|tr,|}ntdi}i}|dD]}|d||d<|d||d<q8g}|D]f}|d} | s\td qO| d d kst| d d kst| d d kst| d d kr|td| qO|dur| d ||dks| d ||dks| d ||dks| d ||dkrtd| qO||qO|S)a Removes invalid predictions from coco result such as: - negative bbox value - extreme bbox value Args: result_list_or_path: path or list for coco result json dataset_dict_or_path (optional): path or dict for coco dataset json z(incorrect type for "result_list_or_path"Nz!incorrect type for "dataset_dict"rrrrr6z+ignoring invalid prediction with empty bboxrrPrQrRz'ignoring invalid prediction with bbox: rM)r;r@rrerLr9rr) rr result_list dataset_dictimage_id_to_heightimage_id_to_widthrfixed_result_list coco_resultr6rrrremove_invalid_coco_resultssD      0 rr7rGr: Coco | Noner;r?r!cCs$tdtt||||||ddS)zpDeprecated. Please use export_coco_as_yolo instead. Calls export_coco_as_yolo with the same arguments. zLexport_coco_as_yolov5 is deprecated. Please use export_coco_as_yolo instead.rGr:r;r?r@rJN)rgrhriexport_coco_as_yolorrrrexport_coco_as_yolov5 s  rcCszddl}Wn tytdw|r|sd}n |r |r d}ntd|r8d|kr3dks8tdtd|rI|j||d }|d }|d }ttj|d } | jddd ttj|d} | jddd t | ||j d|d|s{Jdt | ||j d|dt |  ddt |  ddt |jt|jd} t t|d} t| d} |j| | ddWd| S1swY| S)aExports current COCO dataset in ultralytics/YOLO format. Creates train val folders with image symlinks and txt files and a data yaml file. Args: output_dir: str Export directory. train_coco: Coco coco object for training val_coco: Coco coco object for val train_split_rate: float train split rate between 0 and 1. will be used when val_coco is None. numpy_seed: int To fix the numpy seed. disable_symlink: bool If True, copy images instead of creating symlinks. Returns: yaml_path: str Path for the exported YOLO data.yml rNVPlease run "pip install -U pyyaml" to install yaml first for YOLO formatted exporting.TFz 'train_coco' have to be providedrPrPrQr:r;rRrSrVrWzValidation Coco object not set\/rXr]r^r_)rarbrGrErrrrrcrdrr@rrrrerfrgrh)rGr:r;r?r@rJrarirjrkrlrmrnrorrrr3 sh       ryml_pathcCs"tdtt|||||ddS)zDeprecated. Please use export_coco_as_yolo_via_yml instead. Calls export_coco_as_yolo_via_yml with the same arguments. z\export_coco_as_yolov5_via_yml is deprecated. Please use export_coco_as_yolo_via_yml instead.r#rGr?r@rJN)rgrhriexport_coco_as_yolo_via_ymlr$rrrexport_coco_as_yolov5_via_yml s r&c Cszddl}Wn tytdwt| }||}Wdn1s&wY|drF|ds:t|dtj|d|dd}nd}|drc|d sWt|d tj|d|d d} nd} t||| |||d } | S) a&Exports current COCO dataset in ultralytics/YOLO format. Creates train val folders with image symlinks and txt files and a data yaml file. Uses a yml file as input. Args: yml_path: str file should contain these fields: train_json_path: str train_image_dir: str val_json_path: str val_image_dir: str output_dir: str Export directory. train_split_rate: float train split rate between 0 and 1. will be used when val_json_path is None. numpy_seed: int To fix the numpy seed. disable_symlink: bool If True, copy images instead of creating symlinks. Returns: yaml_path: str Path for the exported YOLO data.yml rNr train_json_pathtrain_image_dirz is missing `train_image_dir`r val_json_path val_image_dirz is missing `val_image_dir`r)rarbrg safe_loadrGrrr) r#rGr?r@rJrastream config_dictr:r;rnrrrr% s:      r%)FFF)FF)rr9rr9rr9)rr9rr@rr@rrr)rr9rr9rrrr9r)rr@rr@rr@)rr9rr)Fr)rSrSTT) rr@rr@rrrrrr9)rrrr)NNr7rF)rGr@r:rr;rr?r!)r7rF)r#r@rGr@r?r!)< __future__rrrr<rrg collectionsrr dataclassesrmultiprocessingrpathlibrrr typingr numpyrr sahi.loggerr sahi.utils.filer rrsahi.utils.shapelyrrrrr-rrr|rrrrrdrrrrrrrrrrrrrrrrr&r%rrrrs       %sFLWM$ ) V =  1 1  # JA 0 s;  ]