k iAdZddlZddlZddlZddlmZmZmZmZm Z ddl m Z m Z m Z ddlmZddlmZddlmZdd lmZdd lmZejeZeZd ed efd Zded efdZded e fdZGddeZ dS)zSemantic cache middleware for LLM responses. This module provides a middleware that caches LLM responses based on semantic similarity using Redis and vector embeddings. Compatible with LangChain's AgentMiddleware protocol for use with create_agent. N)Any AwaitableCallableListUnion)ModelCallResult ModelRequest ModelResponse) AIMessage) SemanticCache)JsonPlusRedisSerializer)AsyncRedisMiddleware)SemanticCacheConfigcontentreturnct|ts|Sg}|D]b}t|tr6d|vr2|d|DM||c|S)zStrip provider-specific IDs from content blocks. When using the OpenAI Responses API, content is a list of blocks with embedded item IDs (rs_, msg_ prefixes). These must be removed from cached messages to prevent duplicate ID errors. idc&i|]\}}|dk ||S)r).0kvs C:\Users\Dell Inspiron 16\Desktop\tws\AgrotaPowerBi\back-agrota-powerbi\mcp-client-agrota\venv\Lib\site-packages\langgraph/middleware/redis/semantic_cache.py z&_strip_content_ids..,s#IIIdaqDyyQyyy) isinstancelistdictappenditems)rstrippedblocks r_strip_content_idsr$ s gt $ $H## eT " " #tu}} OOIIekkmmIII J J J J OOE " " " " Orresponsec t|drt|jtrt |jD]q}t|dr(t j|cSt|dr't jdt|ddicSrt jddiSt|dr&t j|St|trt j|St|dt|}t jd|iS)zSerialize a model response for cache storage. Uses LangChain's to_json() for proper serialization of LangChain objects. Args: response: The response to serialize. Returns: JSON string representation of the response. resultto_jsonr) hasattrrr'rreversedjsondumpsr(getattrrstr)r%msgrs r_serialize_responser12sMx"" +z(/4'H'H +HO,, L LCsI&& Lz#++--00000i(( Lz9gc9b.I.I"JKKKKK Lz9b/***x##.z(**,,---(D!!$z(###h 3x==99G :y'* + ++r cached_strc ttj} tj|}t |t r|ddvr|ddkrt |}t |trA| |t|j ddiid}t|gd St|d t|}tt||ddi gd S|d d }tt||ddi gd Sttt||ddi gd S#tj$r'tt||ddi gd cYSwxYw)aDeserialize a cached response into a ModelResponse. Uses the project's JsonPlusRedisSerializer for proper LangChain object revival. Always returns a ModelResponse to maintain compatibility with agent routing. IMPORTANT: Each cache hit generates a NEW message ID (UUID). This is critical for frontend streaming compatibility - without unique IDs, the frontend deduplicates messages and cached responses don't appear. The cached response is also marked with additional_kwargs={"cached": True} to allow consumers to identify cached responses. Args: cached_str: The cached JSON string. Returns: A ModelResponse containing the cached message with a unique ID. lc)rtype constructorcachedT)rradditional_kwargsresponse_metadata)updateN)r'structured_responser)rrr9r))r/uuiduuid4r,loadsrrget _serializer_revive_if_neededr model_copyr$rr r.JSONDecodeError)r2new_message_iddatarevivedcached_messagers r_deserialize_responserIVsT*&&NC z*%% dD ! !* xx~~''DHHV,<,< ,M,M%77==gy11 &-%7%7"0'9'/'J'J2:D1A13   &8&&N) ./T"'9c'llCC$!$+-/7.>)-    hhy"--G  ')+3T*:%)    II%'/&6!%          &%'/&6!%      s%C F,AF.9F(0F3GGcLeZdZUdZeed<eed<deddffd ZddZd e e e e e fe fde fd Zd e defd Zd e e e e e fe fde e fd Zd e e e e e fe fdefdZdedeegeefdefdZxZS)SemanticCacheMiddlewarea,Middleware that caches LLM responses based on semantic similarity. Uses redisvl.extensions.llmcache.SemanticCache to store and retrieve cached responses. When a request is semantically similar to a previous request (within the distance threshold), the cached response is returned without calling the LLM. By default, only "final" responses (those without tool_calls) are cached. This prevents caching intermediate responses that require tool execution. Example: ```python from langgraph.middleware.redis import ( SemanticCacheMiddleware, SemanticCacheConfig, ) config = SemanticCacheConfig( redis_url="redis://localhost:6379", distance_threshold=0.1, ttl_seconds=3600, ) middleware = SemanticCacheMiddleware(config) async def call_llm(request): # Your LLM call here return response # Use middleware result = await middleware.awrap_model_call(request, call_llm) ``` _cache_configconfigrNcXt|||_dS)z{Initialize the semantic cache middleware. Args: config: Configuration for the semantic cache. N)super__init__rM)selfrN __class__s rrQz SemanticCacheMiddleware.__init__s(     rc8K|jj|jjd}|jjr|jj|d<n|jjr|jj|d<|jj|jj|d<|jj|jj|d<tdi||_dS)aSet up the SemanticCache instance. Note: SemanticCache from redisvl uses synchronous Redis operations internally, so we must provide redis_url and let it manage its own sync connection rather than passing our async client. )namedistance_threshold redis_urlconnection_kwargsN vectorizerttlr) rMrUrVrWconnection_argsrY ttl_secondsr rL)rR cache_kwargss r _setup_asyncz$SemanticCacheMiddleware._setup_asyncsL%"&,"A( (  < ! M(, (>L % % \ ) M04 0LL, - < " .)-)@L & < # /"&,":L #33l33 rmessagesc8|sdSt|D]}t|tr5|dd}|dkr|ddcSLt |ddpt |dd}|dvrt |ddcSdS)aGExtract the prompt to use for cache lookup. Extracts the last user message content from the messages list. Handles both dict-style messages and LangChain message objects. Args: messages: List of messages from the request. Returns: The extracted prompt string. r)roleuserrr6N)rbhuman)r+rrr@r.)rRr_messageramsg_types r_extract_promptz'SemanticCacheMiddleware._extract_prompts 2 )) ; ;G'4(( ;{{62..6>>";;y"55555"#7FD99WVT>>000"7Ir:::::1rrr%c t|tr|d}nVt|dd}|Ct |dr3|j}|r*t |dkrt|ddd}| S)zCheck if the response is a final response (no tool calls). Args: response: The model response to check (dict or LangChain type). Returns: True if the response is final (should be cached), False otherwise. tool_callsNr'r)rrr@r.r*r'len)rRr%rhr's r_is_final_responsez*SemanticCacheMiddleware._is_final_responses h % % H!l33JJ!<>>J!gh&A&A!!Hc&kkAoo!(L$!G!GJ~rcg}|D]}t|tr\|ddp|dd}|dkr)||ddst |ddpt |dd}|dks|jjdkr$|t |dd|S)zExtract tool names from tool result messages. Args: messages: List of messages from the request. Returns: List of tool names that have results in the messages. rar)r6toolrUN ToolMessage)rrr@r r.rS__name__)rRr_ tool_namesrdrares r_get_tool_names_from_resultsz4SemanticCacheMiddleware._get_tool_names_from_results4s  D DG'4(( D{{62..I'++fb2I2I6>>%%gkk&"&=&=>>>"7FD99WVT>>v%%):)C})T)T%%ggvr&B&BCCCrc||}|sdS|jjdS|D]}|r||jjvrdSdS)aCheck if cache should be skipped due to tool results. When tool results are present, we check if ALL tools are in the deterministic_tools list. If so, caching is safe. Otherwise, we skip the cache to avoid returning stale responses. Args: messages: List of messages from the request. Returns: True if cache should be skipped, False if caching is OK. FNT)rprMdeterministic_tools)rRr_ro tool_names r#_should_skip_cache_for_tool_resultsz;SemanticCacheMiddleware._should_skip_cache_for_tool_resultsMsm66x@@  5 < + 34$  I Ydl.NNNtturrequesthandlercK|d{Vt|tr|dg}nt |dg}||}|s||d{VS||rtd||d{V}|j j r| |ro t|}|j ||d{Vn<#t$r/}|jstd|Yd}~nd}~wwxYw|S |j |d{V}|rR|dd} | r5td |dd d t'| Sn<#t$r/}|jstd |Yd}~nd}~wwxYw||d{V}|j j p| |} | r t|}|j ||d{Vtd |dd d n<#t$r/}|jstd|Yd}~nd}~wwxYw|S)aWrap a model call with semantic caching. Checks the cache for a semantically similar request. If found, returns the cached response. Otherwise, calls the handler and optionally caches the result. Args: request: The model request containing messages. handler: The async function to call the model. Returns: The model response (from cache or handler). Raises: Exception: If graceful_degradation is False and cache operations fail. Nr_z@Skipping cache - request contains non-deterministic tool results)promptr%zCache store failed: )rxrr%zCache hit for prompt: 2z...z%Cache check failed, calling handler: zCached response for prompt: )_ensure_initialized_asyncrrr@r.rfrtloggerdebugrMcache_final_onlyrjr1rLastore Exception_graceful_degradationwarningacheckrI) rRrurvr_rxr% response_strer8cached_response should_caches rawrap_model_callz(SemanticCacheMiddleware.awrap_model_callps*,,......... gt $ $ 8{{:r22HHw B77H%%h// * )))))))) )  3 3H = =  LLR   %WW--------H<0 ?D4K4KH4U4U ??#6x#@#@L+,,F\,RRRRRRRRRR ???5NN#=!#=#=>>>>>>>>?O H;--V-<<<<<<<33F-G(HS@34.U4S>3#678 c2!U4S>3#678! !!!!FTT<.)M*BBCT  TTTTTTTTrrK)!rr,loggingr=typingrrrrr!langchain.agents.middleware.typesrr r langchain_core.messagesr redisvl.extensions.cache.llmr )langgraph.checkpoint.redis.jsonplus_redisr aiortypesr getLoggerrnr{rAr$r/r1rIrKrrrrs  88888888888888 .-----666666MMMMMM%%%%%%&&&&&&  8 $ $&%'' $!,#!,#!,!,!,!,HZ cZ mZ Z Z Z zQQQQQ2QQQQQr