Unleashing the Power of AI: A Comprehensive Guide to Next-Generation Search Engines

An in-depth exploration of AI-driven search pipelines, architecture, and best practices

Table of Contents

• 1. Introduction
• 2. Flowchart Overview
• 3. User Input & Preprocessing
• 4. Feature Extraction (Image, Audio, Text)
• 5. Indexing & Vector Search
• 6. Metadata Store & Hybrid Ranking
• 7. Query Execution & Nearest Neighbor Search
• 8. Multi-modal Fusion Model
• 9. Personalization & Ranking
• 10. Model Hosting & Retrieval Pipeline
• 11. Real-Time Implementation & Scalability
• 12. Monitoring & CI/CD
• 13. Advanced Examples & Use Cases
• 14. Source Code Examples
• 15. Additional Diagram
• 16. Conclusion

1. Introduction

The evolution of information retrieval has reached new heights with the advent of AI-powered search engines. Traditional search engines often rely on keyword matching, Boolean operators, and meticulously curated ranking algorithms that depend heavily on textual features. However, as the volume of data grows and the types of media expand—ranging from text documents to images, audio files, and even real-time streams—the need for more sophisticated methods of understanding and retrieving information becomes evident.

Modern AI-powered search engines leverage deep learning, vector embeddings, hybrid ranking mechanisms, and multi-modal processing to deliver results that are contextually relevant, personalized, and holistic. The flowchart provided (and discussed in detail throughout this article) illustrates the complexity of these systems, showcasing how user input travels through multiple stages—from preprocessing to feature extraction, indexing, ranking, query execution, and beyond.

In this comprehensive guide, we will explore each stage of this flowchart in detail. We will examine the intricacies of vector search engines, discuss how image and audio features are extracted, delve into the concept of hybrid ranking that combines traditional metadata-based approaches with modern neural ranking techniques, and highlight the critical role of personalization in delivering user-centric results. We will also explore real-time implementation considerations, monitoring, continuous integration, and continuous deployment (CI/CD) best practices, as well as advanced use cases that illustrate the remarkable capabilities of AI-driven search.

Whether you are a data scientist, a software engineer, a product manager, or simply an enthusiast interested in the future of information retrieval, this article aims to provide an all-encompassing view of how next-generation search engines are designed, built, and maintained.

By the end, you will have a deep understanding of the core components, practical examples of how to implement various parts of the system, and insights into the architectural considerations necessary to ensure scalability, reliability, and performance.

2. Flowchart Overview

The flowchart in question—an AI-Powered Search Engine Flowchart—depicts a multi-stage pipeline that begins with the user and ends with the final retrieval and ranking of relevant results. At a high level, it outlines the following key components:

• User Input: The query or request entered by the user.
• Preprocessing: Steps taken to clean, tokenize, or otherwise prepare the user’s query for downstream tasks.
• Feature Extraction: The generation of vector embeddings from text, images, and audio to facilitate more nuanced similarity searches.
• Indexing: The process of storing these embeddings in a specialized data structure (often a vector store) for efficient retrieval.
• Hybrid Ranking: The combination of metadata-based signals with deep learning–derived similarity scores.
• Query Execution: Where the user’s query is executed against the vector store and metadata store to retrieve candidate results.
• Multi-modal Fusion: A step that integrates text, images, audio, and other media types into a unified representation.
• Personalization: The layer that tailors results based on user profiles, behavioral data, or preferences.
• Ranking & Refinement: Final ranking adjustments, often informed by machine learning models that consider a variety of signals.
• Results Presentation: How the retrieved results are displayed to the user, often with real-time updates or interactive interfaces.

This high-level pipeline is not only relevant for web search engines but also for enterprise search solutions, e-commerce product search, recommendation systems, and any application requiring intelligent information retrieval.

Throughout this article, we will reference the flowchart and dissect each component. We will also provide examples, source code snippets, and a second diagram to clarify the advanced concepts introduced along the way.

3. User Input & Preprocessing

At the very start of any search pipeline is the user input. This could be a text query typed into a search box, a voice query captured by a microphone, or even an image uploaded to a search system for reverse image lookup. Regardless of the modality, the system must capture the user’s intent in a format that can be processed by downstream components.

Preprocessing often involves several tasks:

• Tokenization: Splitting text into individual words, phrases, or tokens.
• Normalization: Lowercasing, removing punctuation, and standardizing text to a consistent format.
• Stopword Removal: Optional removal of common words (like “and,” “the,” “of”) that might not carry significant meaning.
• Stemming/Lemmatization: Converting words to their base or root form (e.g., “running” to “run”).
• Noise Reduction: Filtering out extraneous signals from audio or removing background objects from images (in more advanced systems).

For voice queries, preprocessing might involve speech-to-text conversion, while for images, it might involve basic resizing or format conversion. The goal is to prepare the input so that subsequent feature extraction models can more easily handle it.

The preprocessing step may also include intent classification, where a preliminary machine learning model predicts the type of query (e.g., informational, navigational, transactional) or identifies the domain (e.g., e-commerce, news, social media) to route the query to the most relevant subsystem.

Below is a simple example of a Python-like pseudocode that demonstrates textual preprocessing steps:

Example: Basic Text Preprocessing

def preprocess_text(query):
    # Lowercase
    query = query.lower()
    
    # Remove punctuation
    query = ''.join(ch for ch in query if ch.isalnum() or ch.isspace())
    
    # Tokenize
    tokens = query.split()
    
    # Remove stopwords (simple example)
    stopwords = {"and", "the", "of", "to", "in", "a", "an"}
    tokens = [t for t in tokens if t not in stopwords]
    
    # Stemming (very naive example)
    # In reality, you'd use something like NLTK or spaCy
    stems = []
    for t in tokens:
        if t.endswith("ing"):
            stems.append(t[:-3])
        else:
            stems.append(t)
    
    return stems

# Usage
query = "Running in the park on a sunny day!"
processed = preprocess_text(query)
print(processed)  # ['runn', 'park', 'on', 'sunny', 'day']
                

This example illustrates how raw text queries can be converted into a tokenized and partially normalized list of terms, ready for further processing. While simplistic, it underscores the fundamental steps any AI-powered search engine might take to prepare textual queries.

4. Feature Extraction (Image, Audio, Text)

Feature extraction is at the heart of AI-driven search. It translates raw data—be it text, images, or audio—into a numerical representation (often called embeddings) that captures semantic or contextual meaning.

Textual Embeddings: Modern techniques often involve transformer-based models (such as those inspired by the Transformer architecture) to encode text into dense vector representations. These vectors are designed so that semantically similar pieces of text map to nearby points in the embedding space. This is crucial for neural information retrieval, where queries and documents are compared via their vector embeddings.

Image Features: For image retrieval, convolutional neural networks (CNNs) or vision transformer models can be used to generate feature vectors. These vectors encapsulate visual patterns, colors, shapes, and other cues that help the system understand visual similarity or detect objects within an image.

Audio Features: Audio data can be processed by converting waveforms into spectrograms or Mel-frequency cepstral coefficients (MFCCs). Deep learning models specialized in audio processing can then transform these representations into embeddings that capture the acoustic and temporal patterns in the audio clip.

Below is an illustrative snippet showing how one might generate embeddings for text using a pseudo-transformer model interface:

Example: Generating Textual Embeddings

class SimpleTransformerEncoder:
    def __init__(self):
        # Imagine this is a pre-trained transformer model loaded from a checkpoint
        self.model = "PreTrainedTransformerModel"
    
    def encode(self, text):
        # In a real scenario, you'd tokenize the text, pass it through the model,
        # and extract the final hidden state or a pooling layer output
        # Here, we'll just simulate with a random vector
        import random
        random_vector = [random.random() for _ in range(768)]
        return random_vector

# Usage
encoder = SimpleTransformerEncoder()
embedding = encoder.encode("What is the weather today?")
print("Embedding length:", len(embedding))  # 768
                

In a production environment, these embeddings are generated en masse for all items in the corpus (documents, images, audio clips, etc.) and stored in an index for fast retrieval. Generating these embeddings can be computationally expensive, so it is often done offline or incrementally as new data arrives.

One key advantage of vector embeddings is their ability to capture nuanced semantic relationships. For instance, textual embeddings can recognize that “running” and “jogging” are closely related concepts, while image embeddings can capture that two images contain similar objects or scenes.

5. Indexing & Vector Search

After we obtain embeddings, the next major step is indexing. Traditional indexing for textual search often involves inverted indexes mapping terms to document IDs. However, in an AI-powered search engine, we often rely on vector indexes to facilitate similarity search.

Vector Index Structures (such as those based on approximate nearest neighbor algorithms) allow us to find the vectors most similar to a query vector efficiently. Common data structures or libraries might use techniques like hierarchical navigable small worlds (HNSW), product quantization, or clustering-based approaches to handle large-scale vector data.

The indexing process involves:

• Batch Embedding Generation: Generating embeddings for all items in the corpus.
• Building the Vector Index: Storing these embeddings in a specialized structure for quick nearest neighbor searches.
• Metadata Association: Linking each vector to its associated metadata (e.g., document title, URL, or categories).
• Periodic Rebuilding or Incremental Updates: Updating the index as new data arrives or old data is removed.

Below is a simplified example of how one might insert embeddings into a pseudo vector store:

Example: Indexing Vectors

class SimpleVectorStore:
    def __init__(self):
        self.vectors = []
        self.metadata = []

    def add_item(self, vector, meta):
        self.vectors.append(vector)
        self.metadata.append(meta)
    
    def search(self, query_vector, top_k=5):
        # A naive search that calculates Euclidean distance
        import math
        distances = []
        for i, v in enumerate(self.vectors):
            dist = math.sqrt(sum((q - r)**2 for q, r in zip(query_vector, v)))
            distances.append((dist, i))
        
        distances.sort(key=lambda x: x[0])  # Sort by distance ascending
        results = []
        for i in range(top_k):
            dist, idx = distances[i]
            results.append({"distance": dist, "metadata": self.metadata[idx]})
        return results

# Usage
vector_store = SimpleVectorStore()
# Suppose we have precomputed embeddings for multiple documents
vector_store.add_item([0.1, 0.2, 0.3], {"doc_id": 1, "title": "Doc 1"})
vector_store.add_item([0.2, 0.1, 0.4], {"doc_id": 2, "title": "Doc 2"})
# ...
query_vec = [0.15, 0.15, 0.35]
search_results = vector_store.search(query_vec)
for res in search_results:
    print(res)
                

In real-world scenarios, more advanced data structures are used to handle millions or billions of vectors. Approximate nearest neighbor (ANN) search algorithms reduce query latency and memory usage by allowing small trade-offs in recall for big gains in speed. Libraries and frameworks that provide these capabilities can be integrated into your search engine architecture, but the core principle remains the same: store vectors in a specialized index and quickly retrieve the closest matches for a query vector.

6. Metadata Store & Hybrid Ranking

While vector embeddings capture semantic meaning, traditional metadata often remains vital. Metadata might include titles, timestamps, categories, authors, user ratings, or other structured fields. A robust AI-powered search engine typically stores this metadata in a relational database, NoSQL store, or specialized metadata store for quick lookups and filtering.

Hybrid Ranking merges the best of both worlds:

• Semantic Similarity from vector embeddings.
• Metadata Relevance from structured fields, boosting or filtering results based on specific criteria (e.g., publication date, user rating, or domain authority).

An example scenario might be an e-commerce search engine. When a user searches for “running shoes,” the vector embeddings help find products semantically related to running footwear, while metadata can ensure that only in-stock items are shown, or items from a specific brand are prioritized.

Below is a conceptual snippet that shows how hybrid ranking might be applied:

Example: Hybrid Ranking

def hybrid_ranking(vector_results, metadata_boost=None):
    """
    vector_results: List of dicts with keys: 'distance', 'metadata'
    metadata_boost: Dict of metadata keys and their weight. e.g. {'brand': {'Nike': 1.2}}
    """
    if metadata_boost is None:
        metadata_boost = {}
    
    ranked_results = []
    for result in vector_results:
        dist = result['distance']
        meta = result['metadata']
        
        # Convert distance to a similarity score (inverse)
        similarity_score = 1 / (1 + dist)
        
        # Apply metadata-based boosts
        final_score = similarity_score
        for key, boost_dict in metadata_boost.items():
            if key in meta and meta[key] in boost_dict:
                final_score *= boost_dict[meta[key]]
        
        ranked_results.append({
            "metadata": meta,
            "distance": dist,
            "score": final_score
        })
    
    # Sort by final score descending
    ranked_results.sort(key=lambda x: x['score'], reverse=True)
    return ranked_results
                

This simple illustration shows how a similarity score can be combined with metadata-based boosts to achieve a more nuanced ranking. Real-world systems might incorporate advanced learning-to-rank models, re-ranking pipelines, or query expansion techniques to further refine the final list of results.

7. Query Execution & Nearest Neighbor Search

Once the user submits a query, the system orchestrates several steps in quick succession:

1. Preprocessing: As discussed earlier, the query is cleaned, tokenized, or transformed into a suitable representation.
2. Embedding Generation: The query is passed through the same embedding model used for the corpus, ensuring the vectors are comparable in the same space.
3. Nearest Neighbor Search: The query embedding is matched against the index of item embeddings. Approximate nearest neighbor algorithms may be used to speed up this process.
4. Metadata Filtering: Based on user preferences or domain requirements, some results may be filtered out or given a higher rank.
5. Hybrid Ranking: The final ranking is computed, potentially combining vector similarity scores with metadata signals.
6. Result Presentation: The results are returned to the user, often formatted with additional context or dynamic UI elements.

The Nearest Neighbor Search is particularly critical. Efficient retrieval in large-scale systems requires data structures optimized for high-dimensional spaces, as naive methods scale poorly. Implementations can vary from CPU-based to GPU-accelerated solutions, depending on the required throughput and latency constraints.

Some advanced systems incorporate query-time fusion, where textual, image, and audio embeddings are combined or used in parallel to handle multi-modal queries (e.g., “Find me pictures of beaches where I can surf in California”).

8. Multi-modal Fusion Model

In today’s data-rich world, users often interact with search engines in ways that transcend traditional text queries. They might upload images, provide voice queries, or even specify location-based constraints. A multi-modal fusion model seeks to unify these diverse data types into a single, coherent representation.

The multi-modal fusion process can occur at different stages:

• Early Fusion: Raw data from different modalities is combined and processed together by a single model. For example, an image’s pixel data might be combined with its caption text early in the network.
• Late Fusion: Separate models process each modality independently, generating embeddings or feature vectors. These vectors are then concatenated or otherwise combined in the final layers.
• Intermediate Fusion: A hybrid approach where partial features are shared between modalities at various network depths.

For instance, if a user uploads an image of a product and types a query like “Show me similar items but in red,” the multi-modal fusion model can interpret the image’s features (shape, style, brand) and the text query’s additional constraints (color preference). The search system then retrieves items that visually resemble the uploaded product but filters or re-ranks them based on color.

Multi-modal fusion often requires specialized training data that pairs different modalities (e.g., images with textual descriptions, audio clips with transcripts). The complexity of training and the size of the model can be significant, but the payoff is a more intuitive and user-friendly search experience.

9. Personalization & Ranking

Personalization is a powerful differentiator in modern search engines. Instead of offering the same results to every user, the system adapts to individual preferences, search history, and context. This might involve:

• User Profiles: Storing information about each user’s behavior, interests, and previous interactions.
• Session Context: Taking into account the user’s recent queries or clicked results within the same session.
• Demographic Data: If available and ethically collected, demographic information can help tailor results.
• Geolocation: Showing region-specific results or local recommendations.

Ranking models can incorporate these personalization signals. For instance, if a user frequently searches for technology news, the system might boost tech-related articles in subsequent searches. If the user has a preference for certain brands, those brand items may appear higher in product search results.

Machine Learning-based Ranking typically involves training a model on historical search logs, user clicks, and dwell time metrics. The model learns which features (both embedding-based and metadata-based) correlate with user satisfaction. During query-time, the model scores each candidate result, and a final sorted list is produced.

This section underscores the importance of balancing personalization with diversity. Over-personalization can lead to filter bubbles, where users only see content aligned with their existing preferences. Many systems implement strategies to inject diverse results, ensuring users are exposed to new or alternative information.

10. Model Hosting & Retrieval Pipeline

Hosting the various models used in an AI-powered search engine (e.g., embedding models, ranking models, fusion models) requires careful planning. These models can be large, sometimes spanning hundreds of millions or billions of parameters.

Key considerations include:

• GPU vs. CPU Serving: Depending on the throughput and latency requirements, you may serve models on GPUs for faster inference or on CPUs for cost savings.
• Load Balancing: High-traffic search engines need multiple instances of each model to handle concurrent requests. Load balancers distribute incoming queries among these instances.
• Batching: Combining multiple queries into a single batch can improve GPU utilization, but it also introduces a small delay. Tuning batch sizes is crucial for optimal performance.
• Latency Constraints: If your search engine aims to return results within a fraction of a second, each step in the pipeline must be highly optimized.
• Caching: Caching popular queries or partial computations can drastically reduce latency.

The retrieval pipeline orchestrates the flow from the front-end request to the final results. It typically involves:

1. Receiving the Query: The front-end or API layer passes the query to the retrieval service.
2. Embedding Generation: The query is encoded by the embedding model.
3. Vector Search: The vector index is queried to find the nearest neighbors.
4. Metadata Lookup: The system fetches relevant metadata from a store.
5. Ranking Model: A specialized model or set of heuristics re-ranks the results.
6. Personalization: The results are adjusted based on user preferences.
7. Formatting & Return: The final list is formatted and returned to the user.

Each of these steps must be carefully monitored and optimized to ensure a seamless user experience.

11. Real-Time Implementation & Scalability

Achieving real-time or near-real-time performance in an AI-powered search engine is a challenging task. Large-scale systems must handle thousands or even millions of queries per second while maintaining sub-second response times.

Scalability Strategies:

• Sharding: Splitting the vector index across multiple nodes. Each node handles a portion of the data, and a coordinator node aggregates the results.
• Replication: Duplicating data across multiple nodes for high availability and to distribute query load.
• Asynchronous Updates: Updates to the index might be queued or batched to avoid blocking real-time queries.
• Distributed Query Execution: Queries can be routed to multiple nodes simultaneously, and partial results are merged at a central node.
• Efficient Data Transfer: Minimizing network overhead and employing compression techniques to handle large volumes of vector data.

Systems often integrate a combination of these techniques to meet the demands of large user bases. Additionally, advanced caching layers and real-time streaming architectures (e.g., using event-driven pipelines) help manage continuous data ingestion without compromising query performance.

Real-time indexing (also known as incremental indexing) is another challenge. When new data arrives, the system must generate embeddings and update the vector index without significant downtime. Some solutions maintain a separate “live” index for recent data and periodically merge it with the main index to ensure freshness.

In domains like social media or news, real-time performance is crucial because users expect the latest content to appear in their search results. In e-commerce, it’s important to reflect inventory changes and new product listings quickly to avoid customer frustration.

12. Monitoring & CI/CD

Deploying and maintaining an AI-powered search engine is not a one-time effort. Continuous monitoring ensures the system remains healthy, while CI/CD pipelines facilitate iterative improvements and quick deployment of updates.

Monitoring typically involves:

• Latency Tracking: Measuring query response times at each stage (embedding generation, vector search, ranking, etc.).
• Throughput Analysis: Monitoring the volume of queries handled per second and ensuring it meets service-level agreements (SLAs).
• Quality Metrics: Observing user engagement metrics like click-through rate (CTR), dwell time, and satisfaction scores to gauge search quality.
• Error Rates: Tracking exceptions, timeouts, or index failures.

CI/CD (Continuous Integration/Continuous Deployment) is essential for agile development. It involves:

• Automated Testing: Each commit triggers tests that validate the functionality and performance of the system.
• Model Validation: New models are evaluated against offline and online metrics to ensure they outperform or at least match the existing models.
• Canary Releases: Deploying updates to a small subset of users before rolling them out to the entire user base.
• Rollback Mechanisms: Quickly reverting to a previous version if a new deployment causes issues.

These processes ensure that improvements can be delivered rapidly without sacrificing stability or user satisfaction.

13. Advanced Examples & Use Cases

AI-powered search engines are versatile and can be applied to various domains. Below are a few advanced examples:

1. Voice-Activated Personal Assistants

Voice queries require real-time speech-to-text conversion and advanced natural language understanding to interpret user intent. The system must handle ambiguous phrases, disfluencies, and background noise, all while delivering fast, accurate results.

2. Visual Search in E-Commerce

Users can upload a photo of an item they like, and the system retrieves similar products. This relies heavily on image embeddings and a robust vector search index. Coupled with personalization, the engine might prioritize products that align with the user’s size, brand preferences, or price range.

3. Legal Document Discovery

Large law firms often manage massive repositories of legal documents. AI-powered search engines can semantically index these documents, enabling lawyers to find relevant case law or precedents by describing the scenario in natural language, rather than relying solely on keywords.

4. Medical Image Retrieval

In healthcare, radiologists may want to compare a patient’s scan with thousands of historical images to find similar cases. Multi-modal models that combine text (patient history) and images (scans) can greatly assist in diagnosis and research.

5. Multimedia News Aggregation

News platforms often serve articles, videos, and podcasts. A multi-modal search engine can index all these formats, providing a unified interface where users can find text-based articles, video coverage, and audio interviews on the same topic.

These examples illustrate the broad applicability of AI-powered search. As data types diversify and user expectations grow, the ability to handle text, images, audio, and more within a single cohesive system becomes increasingly critical.

14. Source Code Examples

In this section, we will provide a more holistic example that ties together multiple components of our AI-powered search engine pipeline. The aim is to demonstrate how one might structure the code to handle query embedding, vector search, metadata retrieval, and final ranking in a simplified manner.

End-to-End Pseudocode

class AIPoweredSearchEngine:
    def __init__(self, embedding_model, vector_store, ranking_model, metadata_store):
        self.embedding_model = embedding_model
        self.vector_store = vector_store
        self.ranking_model = ranking_model
        self.metadata_store = metadata_store

    def search(self, query, user_profile=None, top_k=10):
        # Step 1: Preprocess Query
        processed_query = preprocess_text(query)
        
        # Step 2: Generate Embedding
        query_vector = self.embedding_model.encode(" ".join(processed_query))
        
        # Step 3: Vector Search
        raw_results = self.vector_store.search(query_vector, top_k=top_k)
        
        # Step 4: Retrieve Metadata
        enriched_results = []
        for res in raw_results:
            meta_info = self.metadata_store.get_metadata(res['metadata']['doc_id'])
            enriched_results.append({
                "distance": res['distance'],
                "vector_metadata": res['metadata'],
                "meta_info": meta_info
            })
        
        # Step 5: Ranking
        final_ranked = self.ranking_model.rank(enriched_results, user_profile=user_profile)
        
        # Step 6: Return Results
        return final_ranked

# Example usage
embedding_model = SimpleTransformerEncoder()
vector_store = SimpleVectorStore()
ranking_model = SomeAdvancedRankingModel()
metadata_store = SomeMetadataStore()

search_engine = AIPoweredSearchEngine(
    embedding_model,
    vector_store,
    ranking_model,
    metadata_store
)

results = search_engine.search("Find articles on deep learning in healthcare", user_profile={"interests": ["AI", "Health"]})
for r in results:
    print(r)
                

This high-level pseudocode outlines how different components might be stitched together. In a production environment, each component would be far more complex, featuring robust error handling, caching layers, distributed processing, and advanced logging/monitoring.

15. Additional Diagram

Below is a diagram container that showcases a simplified representation of a multi-modal retrieval pipeline, complementing the primary flowchart. This diagram focuses on how text, images, and audio might flow through an AI-powered system, eventually merging in a late-fusion step for unified retrieval.

As depicted, each modality (text, image, audio) is processed by its respective embedding model. The resulting vectors are stored in a unified vector index. During query time, relevant embeddings are retrieved and combined in a late fusion step before final ranking.

16. Conclusion

AI-powered search engines represent a monumental leap forward from traditional keyword-based systems. By leveraging vector embeddings, multi-modal fusion, hybrid ranking strategies, and personalization, these systems deliver highly relevant, context-aware results across diverse data types.

The flowchart we’ve explored provides a high-level roadmap for constructing such a system. From user input and preprocessing to feature extraction, indexing, and advanced ranking, each component plays a crucial role in delivering an optimal search experience. Real-time performance, scalability, and continuous improvement through monitoring and CI/CD pipelines ensure that these systems can meet the demands of modern applications and evolving user expectations.

As technology continues to advance, we can expect AI-powered search to become even more intuitive, personalized, and capable of handling an ever-expanding range of data modalities. Whether in e-commerce, healthcare, legal, or social media, the principles and architecture outlined in this guide serve as a foundation for building cutting-edge search solutions.

We hope this article—replete with examples, diagrams, and source code snippets—has provided valuable insights into the design and implementation of next-generation search engines. The future of information retrieval is bright, and AI lies at the very heart of it.