Migrating from S3
The migration API moves all data from a source AWS S3 bucket to a destination Telnyx Storage bucket without the user incurring a data egress charge by AWS.
Feature
The owner of the AWS account does not get charged by AWS on data transfer to Telnyx when this API is employed.
Achieving minimal costs
There are 3 components to this data pipeline
- User’s AWS S3 bucket in AWS Region X.
- Telnyx’s migration engine in AWS Region X.
- Telnyx’s direct connects with AWS.
Cost minimization is achieved via the following billing practices by AWS.
Intra-region data transfer between S3 and EC2, within the same account, or across different accounts, is free of charge.
Source: AWS S3 Pricing
Source: AWS S3 FAQ
Hence, depending on the region of the source AWS S3 bucket, the API will select the co-located migration engines to best take advantage of this billing practice.
Data Transfer Out (DTO) over AWS Direct Connect within the same geopolitical region is heavily discounted in comparison to DTO over the internet.
Source: AWS Direct Connect
Telnyx’s infrastructure is multi-cloud and multi-region with PoPs in multiple geopolitical regions. AWS Direct Connect is one of the components of that architecture. As a result, the migration API takes advantage of the discounted rate of DTO within the same geopolitical region to move data off AWS into Telnyx.
AWS S3 vs Telnyx Storage Costs Revisited
Assume a user has the following pattern in us-east-2 Ohio.
Ignoring API operations since those costs are marginal, this is their costs breakdown. In the Appendix, you can see these costs are corroborated by AWS Cost Calculator.
With the migration API, data can be moved to Telnyx without egress charge from AWS. The post migration costs are as follows.
We are offering this API free of charge to users in its beta stage.
In the future, we will require minimum storage duration to offset the costs we incur with AWS for DTO over Direct Connect.
API Concepts and Procedure
Coverage
This API shows you the supported AWS S3 regions. Prior to using the API for migration, ensure the AWS S3 bucket you want to migrate is among the supported regions.
curl --location 'https://api.telnyx.com/v2/storage/migration_source_coverage' \
--header 'Authorization: Bearer XXX'
{
"data": [
{
"provider": "aws",
"source_region": "us-west-1"
},
{
"provider": "aws",
"source_region": "us-east-1"
},
{
"provider": "aws",
"source_region": "us-east-2"
}
]
}
Migration Sources
Only standard class is supported. Restore data in glacier before attempting migration.
This API allows you to define the source bucket in AWS. In order to use this API, you need to provide it with a pair of AWS access key and secret access key. We advise you to create an IAM role with a Read Only user for this purpose.
curl --location ‘https://api.telnyx.com/v2/storage/migration_sources’ \
--header ‘Content-Type: application/json’ \
--header ‘Authorization: Bearer XXX’ \
--data ‘{
“provider”: “aws”,
“provider_auth”: {
“access_key”: “XXX”,
“secret_access_key”: “XXX”
},
“bucket_name”: “source-west-bucket-demo”
}’
{
“data”: {
“id”: “48f215e7-8f16-4e65-aa31-9340d4a18745”,
“provider”: “aws”,
“provider_auth”: {
“access_key”: “XXX”,
“secret_access_key”: “XXXXXX”
},
“bucket_name”: “source-west-bucket-demo”,
“source_region”: “us-west-1"
}
}
The following errors might be possible —
- A bucket doesn’t exist.
{
"errors": [
{
"code": "15005",
"title": "Bucket does not exist",
"detail": "Bucket does not exist."
}
]
}
- A bucket you don’t own.
{
"errors": [
{
"code": "15025",
"title": "Access denied",
"detail": "Access denied reading migration source bucket."
}
]
}
- A bucket in a region not under coverage.
{
"errors": [
{
"code": "15003",
"title": "Bucket region invalid",
"detail": "You have provided an invalid bucket region."
}
]
}
Migrations
Lastly, you can create a migration. If the target bucket doesn’t exist, the API will attempt to create it for you. If the desired bucket name is not available or invalid, you will receive an error right away.
In addition, you do not have to match source bucket region to target bucket region; in other words, you can migrate data from an AWS source bucket in us-west-1 to Telnyx target bucket in us-east-1. You will not be charged for DTO by AWS as the API will use a migration engine in us-west-1.
When the refresh parameter is set to false, a one time migration will be created. Otherwise, the API will periodically synchronize the source and destination bucket.
curl --location 'https://api.telnyx.com/v2/storage/migrations' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer XXX' \
--data '{
"source_id": "1c73c8d9-d65a-4f61-ab41-afa095324c5d",
"target_bucket_name": "target-bucket-test-account",
"target_region": "us-west-1",
"refresh": false
}'
{
"data": {
"id": "04532400-7d40-4862-8437-17937d48b405",
"source_id": "1c73c8d9-d65a-4f61-ab41-afa095324c5d",
"target_bucket_name": "target-bucket-test-account",
"target_region": "us-west-1",
"refresh": false,
"last_copy": "0001-01-01T00:00:00Z",
"status": "pending",
"bytes_to_migrate": 0,
"bytes_migrated": 0,
"speed": 0,
"eta": "2024-05-10T20:47:33.821604511Z",
"created_at": "2024-05-10T20:47:33.756241Z"
}
}
Checking Migration Progress
You can periodically poll the API to see its status.
curl --location 'https://api.telnyx.com/v2/storage/migrations/04532400-7d40-4862-8437-17937d48b405' \
--header 'Authorization: Bearer XXX'
{
"data": {
"id": "04532400-7d40-4862-8437-17937d48b405",
"source_id": "1c73c8d9-d65a-4f61-ab41-afa095324c5d",
"target_bucket_name": "target-bucket-test-account",
"target_region": "us-west-1",
"refresh": false,
"last_copy": "2024-05-10T21:07:27.43006Z",
"status": "complete",
"bytes_to_migrate": 10485760000,
"bytes_migrated": 10485760000,
"speed": 7226700,
"eta": "0001-01-01T00:00:00Z",
"created_at": "2024-05-10T20:47:33.756241Z"
}
}
Alternatively, check in on the metric of the target bucket. When it’s reached the expected size or all the objects appear in there, the migration is complete.
