Q i3ddlZddlmZddlmZmZmZmZmZm Z ddl m Z ddl m Z mZmZddlmZerddlmZddlmZGd d eZdS) N)cached_property) TYPE_CHECKINGAnyDictListOptionalUnion) ConfigDict)retrystop_after_attemptwait_random_exponential)retry_if_not_exception_type)EmbeddingsCache)BaseVectorizerc eZdZdZedZ d$dedeed ed ed ffd Z e d e fdZ e dZded eeeeffdZded eeeeffdZdeefdZdeefdZd efdZeeddedeeefded eefdZ eeddedeeef d%deed ed eeefd!Z!ded eeefffd" Z"e d efd#Z#xZ$S)&VertexAIVectorizera The VertexAIVectorizer uses Google's VertexAI embedding model API to create embeddings. This vectorizer is tailored for use in environments where integration with Google Cloud Platform (GCP) services is a key requirement. Utilizing this vectorizer requires an active GCP project and location (region), along with appropriate application credentials. These can be provided through the `api_config` dictionary or set the GOOGLE_APPLICATION_CREDENTIALS env var. Additionally, the vertexai python client must be installed with `pip install google-cloud-aiplatform>=1.26`. You can optionally enable caching to improve performance when generating embeddings for repeated inputs. .. code-block:: python # Basic usage vectorizer = VertexAIVectorizer( model="textembedding-gecko", api_config={ "project_id": "your_gcp_project_id", # OR set GCP_PROJECT_ID "location": "your_gcp_location", # OR set GCP_LOCATION }) embedding = vectorizer.embed("Hello, world!") # With caching enabled from redisvl.extensions.cache.embeddings import EmbeddingsCache cache = EmbeddingsCache(name="vertexai_embeddings_cache") vectorizer = VertexAIVectorizer( model="textembedding-gecko", api_config={ "project_id": "your_gcp_project_id", "location": "your_gcp_location", }, cache=cache ) # First call will compute and cache the embedding embedding1 = vectorizer.embed("Hello, world!") # Second call will retrieve from cache embedding2 = vectorizer.embed("Hello, world!") # Batch embedding of multiple texts embeddings = vectorizer.embed_many( ["Hello, world!", "Goodbye, world!"], batch_size=2 ) # Multimodal usage from vertexai.vision_models import Image, Video vectorizer = VertexAIVectorizer( model="multimodalembedding@001", api_config={ "project_id": "your_gcp_project_id", # OR set GCP_PROJECT_ID "location": "your_gcp_location", # OR set GCP_LOCATION } ) text_embedding = vectorizer.embed("Hello, world!") image_embedding = vectorizer.embed(Image.load_from_file("path/to/your/image.jpg")) video_embedding = vectorizer.embed(Video.load_from_file("path/to/your/video.mp4")) T)arbitrary_types_allowedtextembedding-geckoNfloat32model api_configdtypecacherc lt||||j|fi|dS)aInitialize the VertexAI vectorizer. Args: model (str): Model to use for embedding. Defaults to 'textembedding-gecko'. api_config (Optional[Dict], optional): Dictionary containing the API config details. Defaults to None. dtype (str): the default datatype to use when embedding text as byte arrays. Used when setting `as_buffer=True` in calls to embed() and embed_many(). Defaults to 'float32'. cache (Optional[EmbeddingsCache]): Optional EmbeddingsCache instance to cache embeddings for better performance with repeated texts. Defaults to None. Raises: ImportError: If the google-cloud-aiplatform library is not installed. ValueError: If the API key is not provided. ValueError: If an invalid dtype is provided. )rrrN)super__init___setup)selfrrrrkwargs __class__s C:\Users\Dell Inspiron 16\Desktop\tws\AgrotaPowerBi\back-agrota-powerbi\mcp-client-agrota\venv\Lib\site-packages\redisvl/utils/vectorize/vertexai.pyrzVertexAIVectorizer.__init__VsD4 uE??? J))&)))))returncd|jvS)z/Whether a multimodal model has been configured. multimodal)rrs r! is_multimodalz VertexAIVectorizer.is_multimodaltstz))r"c|jr ddlm}||jSddlm}||jS)z3Get the appropriate client based on the model type.r)MultiModalEmbeddingModel)TextEmbeddingModel)r'vertexai.vision_modelsr)from_pretrainedrvertexai.language_modelsr*)rr)r*s r!_clientzVertexAIVectorizer._clientys`   H G G G G G G+;;DJGG G??????!11$*===r" image_pathc ||jstdddlm}|j||fi|S)zIEmbed an image (from its path on disk) using a VertexAI multimodal model.z/Cannot embed image with a non-multimodal model.r)Image)r' ValueErrorr+r1embedload_from_file)rr/rr1s r! embed_imagezVertexAIVectorizer.embed_imageW! PNOO O000000tz%..z::EEfEEEr" video_pathc ||jstdddlm}|j||fi|S)zHEmbed a video (from its path on disk) using a VertexAI multimodal model.z/Cannot embed video with a non-multimodal model.r)Video)r'r2r+r9r3r4)rr7rr9s r! embed_videozVertexAIVectorizer.embed_videor6r"c T|j|fi|||_dS)zBSet up the VertexAI client and determine the embedding dimensions.N)_initialize_client_set_model_dimsdims)rrrs r!rzVertexAIVectorizer._setups6  55f555((** r"c |r|dntjd}|r|dntjd}|std|std|r|dnd} d dl}|||| dS#t $rt d wxYw) a Setup the VertexAI client using the provided config options or environment variables. Args: api_config: Dictionary with GCP configuration options **kwargs: Additional arguments for initialization Raises: ImportError: If the google-cloud-aiplatform library is not installed ValueError: If required parameters are not provided project_idGCP_PROJECT_IDlocation GCP_LOCATIONzzMissing project_id. Provide the id in the api_config with key 'project_id' or set the GCP_PROJECT_ID environment variable.zMissing location. Provide the location (region) in the api_config with key 'location' or set the GCP_LOCATION environment variable. credentialsNr)projectrBrDzVertexAI vectorizer requires the google-cloud-aiplatform library. Please install with `pip install google-cloud-aiplatform>=1.26`)getosgetenvr2vertexaiinit ImportError)rrrr@rBrDrIs r!r<z%VertexAIVectorizer._initialize_clients(-7 WJNN< ( ( (BIFV> !*% V V VT3r77TTUU U T T TR#a&&RRSS S Ts #&BA B#BB<)minmax)waitstopr contentc l |jrddlm}m}t |t r1|jjd d|i|}|jtd|jSt ||r1|jjd d|i|}|j td|j St ||r<|jjd d|i|}|j td |j dj Std |jj|gfi|djS#t$r}td |d}~wwxYw) a_ Generate a vector embedding for a single input using the VertexAI API. Args: content: Input to embed **kwargs: Additional parameters to pass to the VertexAI API Returns: List[float]: Vector embedding as a list of floats Raises: ValueError: If embedding fails rr1r9contextual_textNz)No text embedding returned from VertexAI.imagez*No image embedding returned from VertexAI.videoz*No video embedding returned from VertexAI.zJInvalid input type for multimodal embedding. Must be str, Image, or Video.zEmbedding input failed: )r'r+r1r9 isinstancerQr.get_embeddingstext_embeddingr2image_embeddingvideo_embeddingsrS TypeErrorvaluesrR)rr]rr1r9resultrUs r!rMzVertexAIVectorizer._embeds&& =!" R????????gs++8T\8(/ F,4()TUUU!00//8T\8% F-5()UVVV!11// 8T\8% F.6()UVVV!215??#8 3t|2G9GGGGJQQ = = =;;;<< < =s,ADADA D$-D D3D..D3 contents batch_sizec |jrtdt|tst d|r*t|dt st d g}|||D]4}|jj|fi|}| d|D5|S#t$r}td|d}~wwxYw)a Generate vector embeddings for a batch of texts using the VertexAI API. Args: contents: List of texts to embed batch_size: Number of texts to process in each API call **kwargs: Additional parameters to pass to the VertexAI API Returns: List[List[float]]: List of vector embeddings as lists of floats Raises: ValueError: If embedding fails zEBatch embedding is not supported for multimodal models with VertexAI.z+Must pass in a list of str values to embed.rcg|] }|j Src)rj).0rs r! z2VertexAIVectorizer._embed_many..Cs">">">18">">">r"zEmbedding texts failed: N) r'NotImplementedErrorrdlistrirQbatchifyr.reextendrRr2)rrmrnr embeddingsbatchresponserUs r! _embed_manyzVertexAIVectorizer._embed_many s,   %W (D)) KIJJ J  KJx{C88 KIJJ J =!Jx<< @ @64<6uGGGG!!">">X">">">????  = = =;;;<< < =s(AB77 CCCcddlm}m}t||r|jSt||r|jSt |S)z&Convert content to a cacheable format.rr_)r+r1r9rd _image_bytes _video_bytesr_serialize_for_cache)rr]r1r9r s r!rz'VertexAIVectorizer._serialize_for_cacheHsi77777777 gu % % (' '  ' ' (' 'ww++G444r"cdS)NrIrcr&s r!typezVertexAIVectorizer.typeRszr")rNrN)rl)%__name__ __module__ __qualname____doc__r model_configrQrrrpropertyboolr'rr.r rfloatbytesr5r:rr<intr=r r r rrir2rrMr{rr __classcell__)r s@r!rrsBBH:d;;;L+%)-1 ***TN* * )* ******<*t***X* > >_ >FcFd5k5>P8QFFFFFcFd5k5>P8QFFFF+$++++1Xd^1111fTTTTT( U $ $ 3 3 3   " "))9j*ABB 4=c4=U 4=4=4=  4=l U $ $ 3 3 3   " "))9j*ABB 68!=!=S !=/2!= d5k !=!=!=  !=F5C5E%*4E555555cXr"r)rG functoolsrtypingrrrrrr pydanticr tenacityr r r tenacity.retryr.redisvl.extensions.cache.embeddings.embeddingsrredisvl.utils.vectorize.baserrrcr"r!rs %%%%%%BBBBBBBBBBBBBBBBGGGGGGGGGG666666ONNNNNN777777EEEEEEEEEEr"