> ## Documentation Index > Fetch the complete documentation index at: https://developers.telnyx.com/llms.txt > Use this file to discover all available pages before exploring further. # Embedding llms-full.txt > Complete machine-readable documentation content for Embedding (AI) for AI agents and LLMs # Telnyx AI: Embedding — Full Documentation > Complete page content for Embedding (AI section) of the Telnyx developer docs ([https://developers.telnyx.com](https://developers.telnyx.com)). > This file: [https://developers.telnyx.com/development/llms/ai-embedding-llms-full-txt.md](https://developers.telnyx.com/development/llms/ai-embedding-llms-full-txt.md) · Root index: [https://developers.telnyx.com/llms.txt](https://developers.telnyx.com/llms.txt) ## Embedding ### Overview > Source: [https://developers.telnyx.com/docs/inference/embedding-rag.md](https://developers.telnyx.com/docs/inference/embedding-rag.md) Embedding and retrieval-augmented generation (RAG) let your applications search prior knowledge before asking a model to respond. Instead of relying solely on a model's training data, RAG retrieves relevant information from your own content and provides it as context at inference time -- producing more accurate, grounded, and up-to-date responses. ## What Is Embedding & RAG? **Embedding** converts text into numeric vectors that capture meaning. Two pieces of text with similar meaning produce similar vectors, even if the exact words differ. This makes it possible to search by concept rather than by keyword. **Retrieval-augmented generation (RAG)** uses those embeddings at query time. When a user asks a question, the system: 1. Embeds the query into a vector 2. Searches your indexed content for the most relevant chunks 3. Passes those chunks as context to a language model 4. The model generates a response grounded in your data The result is answers that reference your actual content -- call transcripts, documents, messages -- rather than guessing from training data. ## What's in This Section Telnyx's managed RAG product. Create searchable collections over your Telnyx communications data and query them with one retrieval API. Search persisted conversation records directly -- the same indexed history that backs Search's conversation sources. Lower-level primitives: embed documents in a Telnyx Storage bucket and run similarity search or clustering over them yourself. Rates for embedding, storage, and search events. Start with [Search](/docs/ai-search) -- it manages chunking, embedding, indexing, and ranking for you. Reach for the [embeddings APIs](/docs/inference/embeddings) when you need the primitives directly; [How Search Works](/docs/ai-search/how-it-works) compares the two. These primitives can be used with AI Assistants, custom agent runtimes, or your own application code. *** ### Pricing > Source: [https://developers.telnyx.com/docs/inference/embedding-rag/pricing.md](https://developers.telnyx.com/docs/inference/embedding-rag/pricing.md) Pricing for the Embedding & RAG section is broken down by feature. Search covers collection management, embedding, storage, and search events. Embedding covers document embedding and storage for use with RAG workflows. Search pricing has three parts: embedding and persisting content when it is indexed for search, storage retention beyond 30 days, and search events when you query a collection. ## Rates | Usage | Price | Notes | | --------------------- | ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Embed and persist | `$0.0015 / 1K characters` | Billed once on input content characters when the content is indexed. Adding an already-indexed source to another collection is free. Includes 30 days of retention. | | Storage after 30 days | `$0.60 / GiB-month` | Applies only when extended retention is enabled. | | Search event | `$0.003 / search` | Charged per ranked search (a request with a `query` parameter). First 10,000 searches per month are free of charge. | | Catalog listing | Free | Browsing a collection's documents without a `query` parameter is not billed. | ## How Billing Works ### Embed and Persist Content is billed once when it is indexed. A source is indexed when its content is first ingested for search; once indexed, the same source can be added to any number of collections at no additional cost. You are not billed for a source simply because it is part of a collection. Billing covers the input content characters and includes 30 days of retention. Storage after 30 days is billed at the extended retention rate only if extended retention is enabled. ### Search Events A **search event** is a single `GET /v2/ai/collections/{slug}/documents` request that includes a `query` parameter. Each ranked search counts as one billable event, regardless of the `top_k` value or the number of results returned. The first 10,000 searches per month are free. Requests without a `query` parameter (catalog listings) are free -- they return documents sorted by date without ranking, and are not counted as search events. ### Multi-Region Fan-Out When a collection's sources are stored across multiple regions, a search query fans out to all relevant regions in parallel. Each region retrieves its matching results independently, and the results are merged into a single ranked list by score. The user does not see or control which regions are searched -- the fan-out and merge happen automatically. Despite querying multiple regions, the entire fan-out counts as **one** billable search event, not one per region. The multi-region behavior does not increase the cost of a search. Region selection is automatic -- there is no region parameter on search requests. ## Example A collection with 10,000 characters of indexed content and one search within the free tier would be priced as: | Line Item | Calculation | Price | | ---------------------- | ----------------------------------------------- | --------- | | Embed and persist | `10K characters * $0.0015` | `$0.015` | | Storage within 30 days | Included | `$0.00` | | One search | Included in the first 10,000 searches per month | `$0.00` | | Total | | `~$0.015` | Embedding pricing covers document embedding and storage for use with RAG workflows. Documents uploaded to a Telnyx Storage bucket are embedded and indexed for similarity search and retrieval. ## Rates | Usage | Price | Notes | | --------------------- | ------------------------- | --------------------------------------------------------------------------------------------------- | | Embed and persist | `$0.0015 / 1K characters` | Billed once on input content characters when documents are embedded. Includes 30 days of retention. | | Storage after 30 days | `$0.60 / GiB-month` | Applies only when extended retention is enabled. | | Similarity search | `$0.003 / search` | Charged per similarity search request. First 10,000 searches per month are free of charge. | ## How Billing Works ### Embed and Persist When you embed documents in a Telnyx Storage bucket, the content is processed into sections and each section is embedded into a vector. You are billed once on the input content characters. This includes 30 days of retention. Storage after 30 days is billed at the extended retention rate only if extended retention is enabled. ### Similarity Search A similarity search request retrieves the most relevant document sections for a given query. Each search counts as one billable event. The first 10,000 searches per month are free. ## Example A bucket with 50,000 characters of document content and 100 searches in a month would be priced as: | Line Item | Calculation | Price | | ---------------------- | ----------------------------------------------- | --------- | | Embed and persist | `50K characters * $0.0015` | `$0.075` | | Storage within 30 days | Included | `$0.00` | | 100 searches | Included in the first 10,000 searches per month | `$0.00` | | Total | | `~$0.075` | *** ## Bucket ### Embeddings > Source: [https://developers.telnyx.com/docs/inference/embeddings.md](https://developers.telnyx.com/docs/inference/embeddings.md) In this tutorial, you'll learn how to: * Upload documents to [Telnyx Storage](https://telnyx.com/products/cloud-storage) * Transform these documents into embeddings, enabling a language model to retrieve relevant sections of your documents * Provide the storage bucket as context for the language model ## Upload your documents You can upload objects to Telnyx's S3-Compatible storage API using our [quickstart](https://developers.telnyx.com/docs/cloud-storage/quick-start) or with our [drag-and-drop interface in the portal](https://portal.telnyx.com/#/storage/buckets). ## Embed your documents Once you've uploaded your documents, you can [embed them via API](https://developers.telnyx.com/api-reference/embeddings/embed-url-content#embed-url-content) or by clicking the "Embed for AI Use" button in the portal while viewing your storage bucket's contents. Behind the scenes, your documents will be processed into sections and each section will be "embedded" based on its contents. Later, when a user asks a language model a question, it will automatically be provided with the most relevant sections of documents from the bucket to help answer the question. ## Chat over your documents Once your documents are embedded, you can try it out in the [AI Playground in the portal](https://portal.telnyx.com/#/ai/playground) by selecting your embedded bucket from the storage dropdown. You can also use embeddings via our [chat completions API](https://developers.telnyx.com/api-reference/openai-chat/create-a-chat-completion-openai-compatible). Here is a Python example. Make sure you have set the `TELNYX_API_KEY` environment variable. Also, update the `question` and `bucket` variables in the sample code. ```python theme={null} import os from openai import OpenAI client = OpenAI( api_key=os.getenv("TELNYX_API_KEY"), base_url="https://api.telnyx.com/v2/ai/openai", ) question = "" bucket = "" chat_completion = client.chat.completions.create( messages=[ { "role": "user", "content": question } ], model="zai-org/GLM-5.2", stream=True, tools=[ { "type": "retrieval", "retrieval": { "bucket_ids": [bucket] } } ] ) for chunk in chat_completion: if chunk.choices[0].delta.content: print(chunk.choices[0].delta.content, end="", flush=True) ``` *** ### Clusters > Source: [https://developers.telnyx.com/docs/inference/clusters.md](https://developers.telnyx.com/docs/inference/clusters.md) In this tutorial, you'll learn: * How [Embeddings](https://developers.telnyx.com/api-reference/embeddings/embed-url-content#embed-url-content) and [Clusters](https://developers.telnyx.com/api-reference/clusters/compute-new-clusters#compute-new-clusters) work * How to leverage them to identify common themes in your data # Embeddings and Clusters Embeddings are numerical representations of concepts within text, image, or audio data. [The representation is a real-valued vector that encodes the meaning of the word in such a way that the words that are closer in the vector space are expected to be similar in meaning](https://en.wikipedia.org/wiki/Word_embedding) Quantifying the semantic similarity of your data opens up several possibilities. For instance, by embedding a Telnyx storage bucket, you can [search for similar content](https://developers.telnyx.com/api-reference/embeddings/embed-url-content#embed-url-content) within your bucket. This tutorial is focused on another application of embeddings: analyzing how your semantic data is [clustered](https://en.wikipedia.org/wiki/Cluster_analysis), which provides insight into common themes and niche subtopics. For example, pictured below are clusters of embeddings computed for the novel The Great Gatsby. Gatsby clusters # Clustering content with Telnyx ## Embed your documents Embedding your content in a Telnyx storage bucket is a prerequisite for computing these clusters. For more information, check out our [Embeddings](https://developers.telnyx.com/docs/inference/embeddings) tutorial. ## Identify clusters Once your documents are embedded, you can [compute clusters](https://developers.telnyx.com/api-reference/clusters/compute-new-clusters#compute-new-clusters) via API. The optional `prefix` and `files` parameters allow you to specfiy a subset of your bucket you would like to cluster. The `min_cluster_size` and `min_subcluster_size` parameters control how clusters are identified. Top-level clusters should be thought of as identifying broad themes in your data. Choose `min_cluster_size` based on the minimum data points you would like to constitute a broader theme. Sub-clusters should be thought of as identifying more specific topics within a broader theme. Choose `min_subcluster_size` based on the minimum data points you would like to constitute a more niche subtopic. ## Identifying themes in The Great Gatsby To demonstrate embedding and clustering a Telnyx storage bucket, we will be using the text from The Great Gatsby. ### Upload to Telnyx Storage You can upload objects to Telnyx's S3-Compatible storage API using our [quickstart](https://developers.telnyx.com/docs/cloud-storage/quick-start) or with our [drag-and-drop interface in the portal](https://portal.telnyx.com/#/storage/buckets). ### Embed your documents Once you've uploaded your documents, you can [embed them via API](https://developers.telnyx.com/api-reference/embeddings/embed-url-content#embed-url-content) or by clicking the "Embed for AI Use" button in the portal while viewing your storage bucket's contents. Behind the scenes, your documents will be processed into chunks and each chunk will be "embedded" based on its contents. Each chunk will be a single data point used in the clustering step. ### Compute clusters You can compute multiple clusterings on the same data. This is helpful to tweak the parameters to find the best clusters for your data. Below is an example API request ``` $ curl --request POST \ --url https://api.telnyx.com/v2/ai/clusters \ --header "Authorization: Bearer $TELNYX_API_KEY" \ --header 'Content-Type: application/json' \ --data '{ "bucket": "cluster-gatsby", "min_cluster_size": 50, "min_subcluster_size": 10 }' ``` And the response ``` {"data":{"task_id":"04dd624f-c9b3-4fc8-8cec-492c8696e9ea"}} ``` ### Inspect clusters You can then take that `task_id` and view the clusters structured as JSON via ``` $ curl --request GET \ --url "https://api.telnyx.com/v2/ai/clusters/04dd624f-c9b3-4fc8-8cec-492c8696e9ea?show_subclusters=true" \ --header "Authorization: Bearer $TELNYX_API_KEY" ``` If you want to see example data from each cluster, you can also pass the `top_n_nodes` query parameter which will include the top N most central data points for each cluster. You can also view a simple graph of the clusters via ``` $ curl --request GET \ --url "https://api.telnyx.com/v2/ai/clusters/04dd624f-c9b3-4fc8-8cec-492c8696e9ea/graph" \ --header "Authorization: Bearer $TELNYX_API_KEY" --output clusters.png ``` If you want to look at a cluster's subclusters, you can pass the `cluster_id` query parameter. Here is a closer look at the sub-clusters related to the cluster for "Daisy's Past" using this endpoint Gatsby clusters The initial parameters can have a large effect on the computed clusters, and the "right" clusters depend heavily on your data set and your goals, so you may have to play around a bit to find what works best. The general idea is that raising `min_cluster_size` will result in broader, more generic clusters. You can also compute as many configurations over your data as you like so you have multiple ways of clustering your data if you'd like. *** ## API Reference (Embedding) ### OpenAI Embeddings * [Create embeddings](https://developers.telnyx.com/api-reference/openai-embeddings/create-embeddings.md): Creates an embedding vector representing the input text. This endpoint is compatible with the OpenAI Embeddings API and may be used with the OpenAI JS or Pytho… * [List embedding models](https://developers.telnyx.com/api-reference/openai-embeddings/list-embedding-models.md): Returns a list of available embedding models. This endpoint is compatible with the OpenAI Models API format. ### Embeddings * [Get Tasks by Status](https://developers.telnyx.com/api-reference/embeddings/get-tasks-by-status.md): Retrieve tasks for the user that are either `queued`, `processing`, `failed`, `success` or `partial_success` based on the query string. Defaults to `queued` an… * [Embed documents](https://developers.telnyx.com/api-reference/embeddings/embed-documents.md): Perform embedding on a Telnyx Storage Bucket using the a embedding model. * [List embedded buckets](https://developers.telnyx.com/api-reference/embeddings/list-embedded-buckets.md): Get all embedding buckets for a user. * [Disable AI for an Embedded Bucket](https://developers.telnyx.com/api-reference/embeddings/disable-ai-for-an-embedded-bucket.md): Deletes an entire bucket's embeddings and disables the bucket for AI-use, returning it to normal storage pricing. * [Get file-level embedding statuses for a bucket](https://developers.telnyx.com/api-reference/embeddings/get-file-level-embedding-statuses-for-a-bucket.md): Get all embedded files for a given user bucket, including their processing status. * [Search for documents](https://developers.telnyx.com/api-reference/embeddings/search-for-documents.md): Perform a similarity search on a Telnyx Storage Bucket, returning the most similar `num_docs` document chunks to the query. * [Embed URL content](https://developers.telnyx.com/api-reference/embeddings/embed-url-content.md): Embed website content from a specified URL, including child pages up to 5 levels deep within the same domain. The process crawls and loads content from the mai… * [Get an embedding task's status](https://developers.telnyx.com/api-reference/embeddings/get-an-embedding-tasks-status.md): Check the status of a current embedding task. Will be one of the following: ### Clusters * [List all clusters](https://developers.telnyx.com/api-reference/clusters/list-all-clusters.md): Retrieve a paginated list of clustering tasks and their statuses. * [Compute new clusters](https://developers.telnyx.com/api-reference/clusters/compute-new-clusters.md): Starts a background task to compute how the data in an embedded storage bucket is clustered. This helps identify common themes and patterns in the data. * [Delete a cluster](https://developers.telnyx.com/api-reference/clusters/delete-a-cluster.md): Delete a clustering task and its computed results. * [Fetch a cluster](https://developers.telnyx.com/api-reference/clusters/fetch-a-cluster.md): Fetch the results of a clustering task, including the discovered clusters. * [Fetch a cluster visualization](https://developers.telnyx.com/api-reference/clusters/fetch-a-cluster-visualization.md): Fetch a visualization image of the clusters computed by a clustering task. ### Conversation Histories * [Search conversation histories](https://developers.telnyx.com/api-reference/conversation-histories/search-conversation-histories.md): Performs semantic vector search across conversation history records.