o Ďi@szdZddlmZddlmZmZeGdddZdeedeee effdd Z d eee efdeefd d Z d S)zEvaluation results utilities for the `.eval_results/*.yaml` format. See https://huggingface.co/docs/hub/eval-results for more details. Specifications are available at https://github.com/huggingface/hub-docs/blob/main/eval_results.yaml. ) dataclass)AnyOptionalc@seZdZUdZeed<eed<eed<dZeeed<dZ eeed<dZ eeed<dZ eeed <dZ eeed <dZ eeed <dZeeed <dZeeed <dddZdS)EvalResultEntrya Evaluation result entry for the `.eval_results/*.yaml` format. Represents evaluation scores stored in model repos that automatically appear on the model page and the benchmark dataset's leaderboard. For the legacy `model-index` format in `README.md`, use [`EvalResult`] instead. See https://huggingface.co/docs/hub/eval-results for more details. Args: dataset_id (`str`): Benchmark dataset ID from the Hub. Example: "cais/hle", "Idavidrein/gpqa". task_id (`str`): Task identifier within the benchmark. Example: "gpqa_diamond". value (`Any`): The metric value. Example: 20.90. dataset_revision (`str`, *optional*): Git SHA of the benchmark dataset. verify_token (`str`, *optional*): A signature that can be used to prove that evaluation is provably auditable and reproducible. date (`str`, *optional*): When the evaluation was run (ISO-8601 datetime). Defaults to git commit time. source_url (`str`, *optional*): Link to the evaluation source (e.g., https://huggingface.co/spaces/SaylorTwift/smollm3-mmlu-pro). Required if `source_name`, `source_user`, or `source_org` is provided. source_name (`str`, *optional*): Display name for the source. Example: "Eval Logs". source_user (`str`, *optional*): HF user name for attribution. Example: "celinah". source_org (`str`, *optional*): HF org name for attribution. Example: "cais". notes (`str`, *optional*): Details about the evaluation setup. Example: "tools", "no-tools", "chain-of-thought". Example: ```python >>> from huggingface_hub import EvalResultEntry >>> # Minimal example with required fields only >>> result = EvalResultEntry( ... dataset_id="Idavidrein/gpqa", ... task_id="gpqa_diamond", ... value=0.412, ... ) >>> # Full example with all fields >>> result = EvalResultEntry( ... dataset_id="cais/hle", ... task_id="default", ... value=20.90, ... dataset_revision="5503434ddd753f426f4b38109466949a1217c2bb", ... verify_token="eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...", ... date="2025-01-15T10:30:00Z", ... source_url="https://huggingface.co/datasets/cais/hle", ... source_name="CAIS HLE", ... source_org="cais", ... notes="no-tools", ... ) ``` dataset_idtask_idvalueNdataset_revision verify_tokendate source_url source_name source_user source_orgnotesreturncCs8|jdus|jdus|jdur|jdurtddSdS)NzaIf `source_name`, `source_user`, or `source_org` is provided, `source_url` must also be provided.)r rrr ValueError)selfrY/home/jeff/fluffinator/venv/lib/python3.10/site-packages/huggingface_hub/_eval_results.py __post_init__Us zEvalResultEntry.__post_init__)rN)__name__ __module__ __qualname____doc__str__annotations__rr rr r r r rrrrrrrrr s <rentriesrcCsg}|D]h}|j|jd}|jdur|j|d<||jd}|jdur'|j|d<|jdur1|j|d<|jdur]d|ji}|jdurE|j|d<|jdurO|j|d <|j durY|j |d <||d <|j durg|j |d <| |q|S) aConvert a list of [`EvalResultEntry`] objects to a YAML-serializable list of dicts. This produces the format expected in `.eval_results/*.yaml` files. Args: entries (`list[EvalResultEntry]`): List of evaluation result entries to serialize. Returns: `list[dict[str, Any]]`: A list of dictionaries ready to be dumped to YAML. Example: ```python >>> from huggingface_hub import EvalResultEntry, eval_result_entries_to_yaml >>> entries = [ ... EvalResultEntry(dataset_id="cais/hle", task_id="default", value=20.90), ... EvalResultEntry(dataset_id="Idavidrein/gpqa", task_id="gpqa_diamond", value=0.412), ... ] >>> yaml_data = eval_result_entries_to_yaml(entries) >>> yaml_data[0] {'dataset': {'id': 'cais/hle', 'task_id': 'default'}, 'value': 20.9} ``` To upload eval results to the Hub: ```python >>> import yaml >>> from huggingface_hub import upload_file, EvalResultEntry, eval_result_entries_to_yaml >>> entries = [ ... EvalResultEntry(dataset_id="cais/hle", task_id="default", value=20.90), ... ] >>> yaml_content = yaml.dump(eval_result_entries_to_yaml(entries)) >>> upload_file( ... path_or_fileobj=yaml_content.encode(), ... path_in_repo=".eval_results/hle.yaml", ... repo_id="your-username/your-model", ... ) ``` )idrNrevision)datasetr verifyTokenr urlnameuserorgsourcer) rrr rr r r r rrrappend)rresultentryr datar&rrreval_result_entries_to_yaml^s.)                  r+r*cCsg}|D]V}|d|}|di}|di}t|d|d|d|d|d|d |r5|d nd |r=|d nd |rE|d nd |rM|dnd |dd }||q|S)aParse a list of dicts into [`EvalResultEntry`] objects. This parses the `.eval_results/*.yaml` format. For the legacy `model-index` format, use [`model_index_to_eval_results`] instead. Args: data (`list[dict[str, Any]]`): A list of dictionaries (e.g., parsed from YAML or API response). Returns: `list[EvalResultEntry]`: A list of evaluation result entry objects. Example: ```python >>> from huggingface_hub import parse_eval_result_entries >>> data = [ ... {"dataset": {"id": "cais/hle", "task_id": "default"}, "value": 20.90}, ... {"dataset": {"id": "Idavidrein/gpqa", "task_id": "gpqa_diamond"}, "value": 0.412}, ... ] >>> entries = parse_eval_result_entries(data) >>> entries[0].dataset_id 'cais/hle' >>> entries[0].value 20.9 ``` r*r r&rrrrr!r r"Nr#r$r%r) rrrr r r r r rrr)getrr')r*ritem entry_datar r&r)rrrparse_eval_result_entriess(    r/N) r dataclassesrtypingrrrlistdictrr+r/rrrrs "R&F