j].dZddlmZddlZddlZddlZddlmZmZm Z m Z m Z e rddl m Z mZmZmZmZddlmZehdZd d dd  d#d Zd$d Zd%d Z d&dZ d'dZdd d(dZd)dZd*dZd+dZd,dZd-dZ d.dZ! d/dZ"ejFdZ$ejFdZ%ejFdejLZ'ejFdZ(ejFdZ)d0dZ*d1d Z+d2d!Z,d3d"Z-y)4uLLM input helper for Azure AI Content Understanding. Provides ``to_llm_input`` — a single public function that converts ``AnalysisResult`` objects into LLM-friendly text (YAML front matter + markdown). ) annotationsN)AnyDictListOptional TYPE_CHECKING)AnalysisContentAnalysisResult ContentFieldDocumentContent DocumentPage) ODataV4Format>pagesfieldscategory timeRange contentType rai_warningsT)include_fieldsinclude_markdownmetadatac  ddlm}t||s!tdt |j t ||jsyt|j}|syddlm t fd|D}g}|D]*}t||||||dkD} | s|j| ,dj|S) a Convert a Content Understanding analysis result into LLM-friendly text. Produces a formatted text string from the analysis result, suitable for injecting into an LLM prompt, storing in a vector database, or passing as tool output. For single-content results (documents, images), the output is a flat text block. For multi-segment results (video, audio), each segment is rendered with its time range. For document classification results (parent with nested segments), the helper automatically expands the parent into per-segment blocks with category labels and markdown slices. :param result: The ``AnalysisResult`` from a Content Understanding analyze operation. :type result: ~azure.ai.contentunderstanding.models.AnalysisResult :keyword include_fields: Whether to include structured fields in the output. Defaults to True. Set to False for markdown-only output (smaller token footprint, no structured data). :paramtype include_fields: bool :keyword include_markdown: Whether to include markdown content in the output. Defaults to True. Set to False for fields-only output. :paramtype include_markdown: bool :keyword metadata: Optional dict of user-supplied key-value pairs to include in the YAML front matter. Common keys include ``"source"`` (filename), ``"department"``, ``"batch_id"``, etc. Metadata keys are placed after ``contentType`` and before auto-detected keys (``timeRange``, ``category``, ``pages``). Metadata keys must not conflict with helper-generated front matter keys. :paramtype metadata: dict[str, Any] or None :returns: A formatted text string with YAML front matter followed by markdown content. :rtype: str :raises TypeError: If *result* is not an ``AnalysisResult``. :raises ValueError: If *metadata* contains a reserved front matter key. Example:: from azure.ai.contentunderstanding import ( ContentUnderstandingClient, to_llm_input, ) from azure.identity import DefaultAzureCredential client = ContentUnderstandingClient(endpoint, DefaultAzureCredential()) poller = client.begin_analyze( analyzer_id="prebuilt-invoice", inputs=[{"source": {"kind": "url", "url": url}}], ) result = poller.result() text = to_llm_input(result) r )r zExpected AnalysisResult, got )AudioVisualContentc3<K|]}t|sdyw)r N) isinstance).0c_AVs g/root/aria/tools/markitdown-venv/lib/python3.12/site-packages/azure/ai/contentunderstanding/_helpers.py zto_llm_input..|s=*Q*<1=s)rrris_multi_segmentz ***** )modelsr r TypeErrortype__name___validate_metadatacontents_get_renderable_contentsrsum_render_content_blockappendjoin) resultrrr_AnalysisResultr)av_countblockscontentblockr s @r! to_llm_inputr50sx: fo .7V 8M8M7NOPPx ??'8H 1=h==HF !%  )-&l    MM%  !    ''c|sytt|jt}|r dj |}t d|dy)zValidate user-supplied front matter metadata. :param metadata: Optional user-supplied metadata. :type metadata: dict[str, Any] or None :raises ValueError: If metadata contains helper-generated front matter keys. N, z0metadata contains reserved front matter key(s): zJ. Use custom keys such as 'source', 'documentId', or 'department' instead.)sortedset intersection_RESERVED_METADATA_KEYSr. ValueError)rreservedkeyss r!r(r(s[ c(m001HIJHyy">tfEW W  r6c`i}|jD]\}}t|}||||<|S)aFlatten CU ContentField objects into plain Python dicts. Recursively resolves ObjectField and ArrayField values using each field's ``.value`` convenience property. :param fields: The ``fields`` dict from an ``AnalysisContent`` object. :type fields: dict[str, ~azure.ai.contentunderstanding.models.ContentField] :returns: A plain dict mapping field names to their resolved values. :rtype: dict[str, Any] )items_resolve_field_value)rresolvednamefieldvals r!_resolve_fieldsrGsA "H||~! e"5) ? HTN! Or6cddlm}m}t||r|j}|r t |SdSt||r=|j }|r.g}|D]!}t|}||j|#|xsdSy|j}|yt|tjtjfr|jS|S)zResolve a single ContentField to a plain Python value. :param field: The content field to resolve. :type field: ~azure.ai.contentunderstanding.models.ContentField :returns: The resolved plain Python value. :rtype: Any r ) ArrayField ObjectFieldN)r$rIrJr value_objectrG value_arrayrBr-valuedatetimedatetime isoformat) rErIrJobjarrrAitemrCrFs r!rBrBs0%%  '*s#44%$ E +/5'LL* + =D  ++C {# x}}56}} Jr6c * ddlm t}|D]D}t| s|js|j s*|j |j Fg}d}|D]}t| rs|jrf|jsYd}|j xsd}|jD]5}|jr|d|jnd}|r||vr*d}|jr|jrt|jtr|jjdd nt|jdd } t|jtr|jjd d nt|jd d } |j| | | z} |j|j|j ||j } |j#| 8|j#||rd fd } |j%| |S)udFilter contents for rendering, handling classification hierarchies. A "parent" content has ``segments`` but no ``category``. When detected, the parent is expanded — each segment becomes a synthetic ``DocumentContent`` with its category, page range, and the corresponding markdown slice extracted via the segment's ``span``. When a segment has a corresponding routed top-level content (linked by ``path`` = ``{parent_path}/{segment_id}``), the top-level item is used instead of the synthetic expansion to preserve extracted fields and avoid duplication. :param contents: The list of analysis contents to filter. :type contents: list[~azure.ai.contentunderstanding.models.AnalysisContent] :returns: The filtered list of renderable contents. :rtype: list[~azure.ai.contentunderstanding.models.AnalysisContent] r r FTr/Noffsetrlength) mime_typestart_page_numberend_page_numbermarkdownrcNt|r|j |jSyNr)rr[)rr s r! _sort_keyz+_get_renderable_contents.._sort_key#s'!_-!2E2E2Q***r6key)r'AnalysisContent'returnint)r$r r:rrpathaddsegments segment_idr]spandictgetgetattrrZr[r\r-sort)r) routed_pathsrr/expanded_classification parent_pathsegseg_pathmdrXrYchildr`r s @r!r*r*s*(L % a )ajjQVV   QVV $%')F#  a )ajj&* #&&,BKzz %@Ck]!CNN+;>2 #BxL(9 !.B~ 'r*LG,,    g /"7B/Bd"R'' r6cj|jrt||j}||ur|St||S)aCAdd ```` markers to document markdown. :param content: The document content with page information. :type content: ~azure.ai.contentunderstanding.models.DocumentContent :param str markdown: The markdown text to annotate. :returns: The markdown with page markers inserted. :rtype: str )r_page_markers_from_spans_page_markers_from_breaks)r3r]r/s r!rr}s5}})(GMMB  !M $Xw 77r6cg}|D]C}|js|j|jdj|jfE|s|S|j dt j dd|}t jd}g|j|D]I}d}|j|jz |z }j|j|fKd fd } g} d} |D]8\} } | | }| j|| || jd| d |} :| j|| d d j| S) aDInsert page markers using ``pages[].spans`` offsets. :param str markdown: The markdown text. :param pages: The list of document pages with span info. :type pages: list[~azure.ai.contentunderstanding.models.DocumentPage] :returns: The markdown with page markers inserted at span offsets. :rtype: str rc |dSr_)ms r!z*_page_markers_from_spans..s qtr6ra\n*\n* c<d}D]\}}||kDs ||z }||z Sr_r)origtotalposdeltashiftss r!_adjusted_offsetz2_page_markers_from_spans.._adjusted_offsets7  JCcz e|r6 Nr)rrerdre) spansr-rX page_numberrnresubcompilefinditerendstartr.)r]rmarkerspagecleaned break_patternrreplacement_lenrrpartsprevrXpage_numadjrs @r!rrsYGE :: NNDJJqM00$2B2BC DE  LL^L$ff0&(CGJJ:;MF  # #H -*!'')#o5 qwwy%()* E D#v& WT#&' z(845   LL 775>r6c d}|j |j}tjd|}g}t|D]4\}}||z}|j }|s|j d|d|6dj |S)aNFall back to splitting on ```` markers. :param str markdown: The markdown text. :param content: The document content with page number info. :type content: ~azure.ai.contentunderstanding.models.DocumentContent :returns: The markdown with page markers replacing PageBreak markers. :rtype: str r rrrr)r[rsplit enumeratestripr-r.) r]r3 start_pagechunksrichunkrtexts r!rrsJ  ,.. XX18 {{}  LL:hZxv> ? @ ;;u r6c0dd}||d||S)uFormat millisecond timestamps as ``MM:SS – MM:SS``. :param int start_ms: Start time in milliseconds. :param int end_ms: End time in milliseconds. :returns: The formatted time range string. :rtype: str c*|dz}|dzdd|dzdS)Ni<02d:r)mstotal_ss r!_fmtz _format_time_range.._fmts**R-$Agl3%788r6u – )rrerdstrr)start_msend_msrs r!r|r|s#98n Xd6l^ 44r6cddlm}t||sy|jr)t d|jD}|r t |S|j }|j}||y||k(r|S|d|S)aReturn a pages value: int for single page, string for range, or *None*. Uses the actual ``pages`` list when available to produce an accurate representation (e.g. ``"2-3, 5"`` instead of ``"2-5"`` when page 4 is absent). Falls back to ``start_page_number`` / ``end_page_number``. :param content: The analysis content to extract page info from. :type content: ~azure.ai.contentunderstanding.models.AnalysisContent r rVNc3NK|]}|j|jyw)N)r)rps r!r"z _format_pages..sXamm>WammXs%%-)r$r rrr9_compress_page_numbersr[r\)r3r numsrrs r!r}r}s( g /}}XW]]XX )$/ /  % %E  ! !C }  | WAcU r6c2|syt|dk(r|dSg}|d}|d}|ddD]7}||dzk(r|}|j||k(r t|n|d||}|}9|j||k(r t|n|d|dj|S)zCompress a sorted list of page numbers into a compact string. Examples: ``[1] -> 1``, ``[1,2,3] -> "1-3"``, ``[2,3,5] -> "2-3, 5"``. :param nums: Sorted list of page numbers. :type nums: list[int] Nr rrr8)lenr-rr.)rrangesrrns r!rr s  4yA~AwF GE 7D !"X q=D MM #e*eWAdV;L MED  MM #e*eWAdV3DE 99V r6cg}|D]l}i}t|ddr|j|d<t|ddr|j|d<t|ddr|j|d<|s\|j |n|S)a4Convert ODataV4Format warnings to plain dicts for YAML output. :param warnings: The list of warnings to convert. :type warnings: list[~azure.ai.contentunderstanding.models.ODataV4Format] :returns: A list of warning dicts with code, message, and target keys. :rtype: list[dict[str, str]] codeNmessagetarget)rmrrrr-)r~rAwentrys r!rr&s#%E    " 1fd #FFE&M 1i & yyE)  1h %hhE(O  LL    Lr6z)^[\-\?\:\,\[\]\{\}\#\&\*\!\|\>\'\"\%\@\`]z [:\#] |[\n\r]z!^(true|false|yes|no|on|off|null)$z)^[+\-]?(\d+\.?\d*|\.\d+)([eE][+\-]?\d+)?$z^\d{4}-\d{2}-\d{2}c^|yt|tr|rdSdSt|tr t|St|trMt j |s t|S|t|k(rtt|St|St|}| xsqtj|xsZtj|xsCtj|xs,tj|xstj|}|rd|jddzdzS|S)zSerialize a scalar to its YAML text form. :param value: The value to serialize. :type value: Any :returns: The YAML-formatted string. :rtype: str nulltruefalse'z'')rboolrerfloatmathisfinite _YAML_BOOLmatch _YAML_NUMBER _YAML_DATE_YAML_SPECIAL_STARTsearch_YAML_SPECIAL_INSIDEreplace)rMs needs_quotes r! _yaml_scalarrIs }%v+G+%5z%}}U#u: "'3u:"5s3u:E3u:E E A  *   A  *   a  *   A  *  % %a (  * & &q ) 2=C!))C& & ,C!Cr6chdg}t||d|jddj|S)zBuild a ``---`` delimited YAML front matter string. :param data: The data to serialize as YAML front matter. :type data: dict[str, Any] :returns: The YAML front matter string. :rtype: str z---r)indentrx) _emit_mappingr-r.)dataliness r!rrhs2wE%a( LL 99U r6c ~d|z}|jD]\}}| t|}t|tr*|s'|j ||dt |||dzNt|t r'|sa|j ||dt||||j ||dt|y)zEmit a YAML mapping (block style). :param lines: The list of output lines to append to. :type lines: list[str] :param mapping: The mapping to emit. :type mapping: dict[str, Any] :param int indent: The current indentation level.  Nrr : )rArrrkr-rlist_emit_sequence)rmappingrprefixrbrMsafe_keys r!rrvsF]FmmoG U = $ eT " LLF8H:Q/ 0 % 3 t $ LLF8H:Q/ 0 5% 0 LLF8H:R U0C/DE FGr6c d|z}|D]}t|trd}|jD]\}}| |r|dn|d}t|} t|tr)|r'|j || dt |||dznZt|t r)|r'|j || dt|||dzn!|j || dt|d}|j |dt|y) aEmit a YAML sequence (block style, compact notation). :param lines: The list of output lines to append to. :type lines: list[str] :param sequence: The sequence to emit. :type sequence: list[Any] :param int indent: The current indentation level. rTNz- rrrF)rrkrArr-rrr) rsequencerrrTfirstkvtagsafe_ks r!rrsF]F< dD !E 19',mVHB-%aa&1LLC5!23!%FQJ74(QLLC5!23"5!VaZ8LLC5<?2C!DE  LLF82l4&8%9: ;%rs # ;;4$ ( !)- [( [([( [( ' [(  [(| 0&% %%ZD%DD\#H H H H  H ' HH H` 8 0f8 5 >6#8!bjj!MN!rzz"23 RZZ G8