Q i'zddlmZmZmZmZmZddlmZerddlm Z ddl m Z de de ddfd ZGd d e ZdS) ) TYPE_CHECKINGAnyCallableListOptional) ConfigDict)EmbeddingsCache)BaseVectorizerresult method_namereturnNc|drt|tr|st|dt|dtr|dst|dt|ddtst|ddSt|tr|st|dt|dtst|ddS) a  Validates the structure of returned embeddings. - For methods named "*_many", expects a list of lists of floats. - For single methods, expects a list of floats. Raises: ValueError: If the embeddings do not match the expected structure. _manyz' must return a non-empty list of lists.rz' must return a list of non-empty lists.z' must return a list of lists of floats.z must return a non-empty list.z must return a list of floats.N)endswith isinstancelist ValueErrorfloat)r r s C:\Users\Dell Inspiron 16\Desktop\tws\AgrotaPowerBi\back-agrota-powerbi\mcp-client-agrota\venv\Lib\site-packages\redisvl/utils/vectorize/custom.py _check_vectorr s3G$$ M&$'' Vv V TTTUU U&)T** V&) V TTTUU U&)A,.. V TTTUU U V V&$'' Mv M KKKLL L&)U++ M KKKLL L M McjeZdZdZedZ ddedeedeed eed ed ed f fd Z dZ e defdZ ddZ dedeefdZ ddeededeeefdZdedeefdZ ddeededeeefdZxZS)CustomVectorizeraThe CustomVectorizer class wraps user-defined embedding methods to create embeddings for data. This vectorizer is designed to accept a provided callable vectorizer and provides a class definition to allow for compatibility with RedisVL. The vectorizer may support both synchronous and asynchronous operations which allows for batch processing of inputs, but at a minimum only synchronous embedding is required to satisfy the 'embed()' method. You can optionally enable caching to improve performance when generating embeddings for repeated inputs. .. code-block:: python # Basic usage with a custom embedding function vectorizer = CustomVectorizer( embed = my_vectorizer.generate_embedding ) embedding = vectorizer.embed("Hello, world!") # With caching enabled from redisvl.extensions.cache.embeddings import EmbeddingsCache cache = EmbeddingsCache(name="my_embeddings_cache") vectorizer = CustomVectorizer( embed=my_vectorizer.generate_embedding, cache=cache ) # First call will compute and cache the embedding embedding1 = vectorizer.embed("Hello, world!") # Second call will retrieve from cache embedding2 = vectorizer.embed("Hello, world!") # Asynchronous batch embedding of multiple texts embeddings = await vectorizer.aembed_many( ["Hello, world!", "How are you?"], batch_size=2 ) T)arbitrary_types_allowedNfloat32embed embed_manyaembed aembed_manydtypecacher c" |d}t|dt|}n$#t$r} td| d} ~ wwxYwt d||||||||dS)aZInitialize the Custom vectorizer. Args: embed (Callable): a Callable function that accepts an object and returns a list of floats. embed_many (Optional[Callable]): a Callable function that accepts a list of objects and returns a list containing lists of floats. Defaults to None. aembed (Optional[Callable]): an asynchronous Callable function that accepts a object and returns a lists of floats. Defaults to None. aembed_many (Optional[Callable]): an asynchronous Callable function that accepts a list of objects and returns a list containing lists of floats. Defaults to None. dtype (str): the default datatype to use when embedding inputs as byte arrays. Used when setting `as_buffer=True` in calls to embed() and embed_many(). Defaults to 'float32'. cache (Optional[EmbeddingsCache]): Optional EmbeddingsCache instance to cache embeddings for better performance with repeated inputs. Defaults to None. Raises: ValueError: if embedding validation fails. zdimension testrz!Failed to validate embed method: Ncustom)modelr dimsr!)rlen Exceptionrsuper__init___setup_functions) selfrrrrr r! test_resultr%e __class__s rr)zCustomVectorizer.__init__Ss4 F% 011K +w / / /{##DD F F FDDDEE E F xu4uMMM eZEEEEEs*. AA  Ac||_||_||_||_d|_d|_|dS)z,Setup the user-provided embedding functions.FN) _embed_func_embed_func_many _aembed_func_aembed_func_many_aembed_validated_aembed_many_validated_validate_optional_funcs)r+rrrrs rr*z!CustomVectorizer._setup_functionszsN  *"!,"'&+# %%'''''rr cdS)Nr#)r+s rtypezCustomVectorizer.typesxrc|jrM |dg}t|ddS#t$r}td|d}~wwxYwdS)z Optionally validate the other user-provided functions if they exist. Raises: ValueError: If any provided function produces invalid results. zdimension test (many)rzInvalid embed_many function: N)r1rr'r)r+ test_batchr-s rr6z)CustomVectorizer._validate_optional_funcss   F F!224K3LMM j,77777 F F F !D!D!DEEE F  F Fs&1 AA  Acontentc l |j|fi|}|S#t$r}td|d}~wwxYw)aaGenerate a vector embedding for a single input using the provided user function. Args: content: Input to embed **kwargs: Additional parameters to pass to the user function Returns: List[float]: Vector embedding as a list of floats Raises: ValueError: If embedding fails Embedding input failed: N)r0r'rr+r<kwargsr r-s r_embedzCustomVectorizer._embeds\ =%T%g8888FM = = =;;;<< < =s 3.3 contents batch_sizec t|tstdjsfd|DS j|fi}|S#t$r}t d|d}~wwxYw)aGenerate vector embeddings for a batch of inputs using the provided user function. Args: contents: List of inputs to embed batch_size: Number of inputs to process in each batch **kwargs: Additional parameters to pass to the user function Returns: List[List[float]]: List of vector embeddings as lists of floats Raises: TypeError: If contents is not a list ValueError: If embedding fails 'Must pass in a list of values to embed.c,g|]}j|fiSr8)rA).0r<r@r+s r z0CustomVectorizer._embed_many..s-KKKwKDK22622KKKrEmbedding inputs failed: N)rr TypeErrorr1r'rr+rCrDr@resultsr-s` ` r _embed_manyzCustomVectorizer._embed_manys"(D)) GEFF F$ LKKKKK(KKK K >+d+H????GN > > ><<<== = >sA A/A**A/cK|js|j|fi|S |j|fi|d{V}|jst|dd|_|S#t$r}t d|d}~wwxYw)aAsynchronously generate a vector embedding for a single input. Args: content: Input to embed **kwargs: Additional parameters to pass to the user async function Returns: List[float]: Vector embedding as a list of floats Raises: NotImplementedError: If no aembed function was provided ValueError: If embedding fails NrTr>)r2rAr4rr'rr?s r_aembedzCustomVectorizer._aembeds  24;w11&11 1 =,4,W??????????F) .fh///)-&M = = =;;;<< < =s3A A.A))A.c,Kt|tstd|js|j||fi|S |j|fi|d{V}|jst |dd|_|S#t$r}td|d}~wwxYw)a+Asynchronously generate vector embeddings for a batch of inputs. Args: contents: List of inputs to embed batch_size: Number of inputs to process in each batch **kwargs: Additional parameters to pass to the user async function Returns: List[List[float]]: List of vector embeddings as lists of floats Raises: TypeError: If contents is not a list NotImplementedError: If no aembed_many function was provided ValueError: If embedding fails rFNrTrJ) rrrKr3rNr5rr'rrLs r _aembed_manyzCustomVectorizer._aembed_manys$(D)) GEFF F% D#4#HjCCFCC C >2D28FFvFFFFFFFFG. 3g}555.2+N > > ><<<== = >s3A22 B<BB)NNNrN)r N)rB)__name__ __module__ __qualname____doc__r model_configrrstrr)r*propertyr9r6rrrrAintrNrPrR __classcell__)r.s@rrr%s))V:d;;;L *.%)*.-1%F%F%FX&%F" %F h' %F  %F)*%F%F%F%F%F%FN ( ( (cX F F F F=c=U ====(68>>S >/2> d5k >>>><=S=tE{====<68">">S ">/2"> d5k ">">">">">">">">rr)typingrrrrrpydanticr.redisvl.extensions.cache.embeddings.embeddingsr redisvl.utils.vectorize.baser rrXrrr8rrr`s??????????????ONNNNNN777777M$MSMTMMMM4f>f>f>f>f>~f>f>f>f>f>r