12.4 C
New York
Monday, March 4, 2024

Deploy Amazon OpenSearch Serverless with Terraform


Amazon OpenSearch Serverless supplies the search and analytical performance of OpenSearch with out the handbook overhead of configuring, managing, and scaling OpenSearch clusters. It routinely scales the assets primarily based in your workload, and also you solely pay for the assets consumed. Managing OpenSearch Serverless is easy, however with infrastructure as code (IaC) software program like Terraform, you’ll be able to simplify your useful resource administration much more.

This submit demonstrates the right way to use Terraform to create, deploy, and clear up OpenSearch Serverless infrastructure.

Answer overview

To create and deploy an OpenSearch Serverless assortment with safety and entry insurance policies utilizing Terraform, you could observe these steps:

  1. Initialize the Terraform configuration.
  2. Create an encryption coverage.
  3. Create an OpenSearch Serverless assortment.
  4. Create a community coverage.
  5. Create a digital personal cloud (VPC) endpoint.
  6. Create an information entry coverage.
  7. Deploy utilizing Terraform.

Stipulations

This submit assumes that you simply’re acquainted with GitHub and Git instructions.

For this walkthrough, you want the next:

Initialize the Terraform configuration

The pattern code is accessible within the Terraform GitHub repo within the terraform-provider-aws/examples/opensearchserverless listing. This configuration will get you began with OpenSearch Serverless. First, clone the repository to your workstation and navigate to the listing:

$ git clone https://github.com/hashicorp/terraform-provider-aws.git &&  
cd ./terraform-provider-aws/examples/opensearchserverless

Initialize the configuration to put in the aws supplier by working the next command:

$ terraform init

The Terraform configuration first defines the model of Terraform required and configures the AWS supplier to launch assets within the Area outlined by the aws_region variable:

# Require Terraform 0.12 or larger
terraform {
  required_version = ">= 0.12"
}

# Set AWS supplier area
supplier "aws" {
  area = var.aws_region
}

The variables used on this Terraform configuration are outlined within the variables.tf file. This submit assumes the default values are used:

variable "aws_region" {
  description = "The AWS area to create issues in."
  default     = "us-east-1"
}

variable "collection_name" {
  description = "Title of the OpenSearch Serverless assortment."
  default     = "example-collection"
}

Create an encryption coverage

Now that the supplier is put in and configured, the Terraform configuration strikes on to defining OpenSearch Serverless insurance policies for safety. OpenSearch Serverless makes use of AWS Key Administration Service (AWS KMS) to encrypt your knowledge. The encryption is managed by an encryption coverage. To create an encryption coverage, use the aws_opensearchserverless_security_policy useful resource, which has a title parameter, a sort of encryption, a JSON string that defines the coverage, and an elective description:

# Creates an encryption safety coverage
useful resource "aws_opensearchserverless_security_policy" "encryption_policy" {
  title        = "example-encryption-policy"
  sort        = "encryption"
  description = "encryption coverage for ${var.collection_name}"
  coverage = jsonencode({
    Guidelines = [
      {
        Resource = [
          "collection/${var.collection_name}"
        ],
        ResourceType = "assortment"
      }
    ],
    AWSOwnedKey = true
  })
}

This encryption coverage is called example-encryption-policy, applies to a group named example-collection, and makes use of an AWS owned key to encrypt the info.

Create an OpenSearch Serverless assortment

You’ll be able to set up your OpenSearch indexes right into a logical grouping known as a assortment. Create a group utilizing the aws_opensearchserverless_collection useful resource, which has a title parameter, and optionally, description, tags, and sort:

# Creates a group
useful resource "aws_opensearchserverless_collection" "assortment" {
  title = var.collection_name

  depends_on = [aws_opensearchserverless_security_policy.encryption_policy]
}

This assortment is called example-collection. If sort is just not specified, a time collection assortment is created. Supported assortment varieties could be discovered within the Terraform documentation for the aws_opensearchserverless_collection useful resource. OpenSearch Serverless requires encryption at relaxation, so an relevant encryption coverage is required earlier than a group could be created. The Terraform configuration explicitly defines this dependency utilizing the depends_on meta-argument. Errors can come up if this dependency is just not outlined.

Now {that a} assortment has been created with an AWS owned KMS key, the Terraform configuration goes on to outline the community and knowledge entry coverage to configure entry to the gathering.

Create a community coverage

A community coverage permits entry to your assortment both over the general public web or by OpenSearch Serverless-managed VPC endpoints. Much like the encryption coverage, to create a community coverage, use the aws_opensearchserverless_security_policy useful resource, which has a title parameter, a sort of community, a JSON string that defines the coverage, and an elective description:

# Creates a community safety coverage
useful resource "aws_opensearchserverless_security_policy" "network_policy" {
  title        = "example-network-policy"
  sort        = "community"
  description = "public entry for dashboard, VPC entry for assortment endpoint"
  coverage = jsonencode([
    {
      Description = "VPC access for collection endpoint",
      Rules = [
        {
          ResourceType = "collection",
          Resource = [
            "collection/${var.collection_name}"
          ]
        }
      ],
      AllowFromPublic = false,
      SourceVPCEs = [
        aws_opensearchserverless_vpc_endpoint.vpc_endpoint.id
      ]
    },
    {
      Description = "Public entry for dashboards",
      Guidelines = [
        {
          ResourceType = "dashboard"
          Resource = [
            "collection/${var.collection_name}"
          ]
        }
      ],
      AllowFromPublic = true
    }
  ])
}

This community coverage is called example-network-policy and applies to the gathering named example-collection. This coverage solely permits entry to the gathering’s OpenSearch endpoint by a VPC endpoint, however permits public entry to the OpenSearch Dashboards endpoint.

You’ll discover the VPC endpoint has not been outlined but, however it’s referenced within the community coverage. Terraform determines this dependency routinely and won’t create the community coverage till the VPC endpoint has been created.

Create a VPC endpoint

A VPC endpoint lets you privately entry your OpenSearch Serverless assortment utilizing AWS PrivateLink (for extra data, confer with Entry AWS companies by AWS PrivateLink). Create a VPC endpoint utilizing the aws_opensearchserverless_vpc_endpoint useful resource, the place you outline title, vpc_id, subnet_ids , and optionally, security_group_ids:

# Creates a VPC endpoint
useful resource "aws_opensearchserverless_vpc_endpoint" "vpc_endpoint" {
  title               = "example-vpc-endpoint"
  vpc_id             = aws_vpc.vpc.id
  subnet_ids         = [aws_subnet.subnet.id]
  security_group_ids = [aws_security_group.security_group.id]
}

Making a VPC and all of the required networking assets is out of scope for this submit, however the minimal required VPC assets are created right here in a separate file to display the VPC endpoint performance. Confer with Getting Began with Amazon VPC to be taught extra.

Create an information entry coverage

The configuration defines a knowledge supply that appears up details about the context Terraform is at the moment working in. This knowledge supply is used when defining the info entry coverage. Extra data could be discovered within the Terraform documentation for aws_caller_identity.

# Will get entry to the efficient Account ID by which Terraform is allowed
knowledge "aws_caller_identity" "present" {}

A knowledge entry coverage permits you to outline who has entry to collections and indexes. The information entry coverage is outlined utilizing the aws_opensearchserverless_access_policy useful resource, which has a title parameter, a sort parameter set to knowledge, a JSON string that defines the coverage, and an elective description:

# Creates an information entry coverage
useful resource "aws_opensearchserverless_access_policy" "data_access_policy" {
  title        = "example-data-access-policy"
  sort        = "knowledge"
  description = "enable index and assortment entry"
  coverage = jsonencode([
    {
      Rules = [
        {
          ResourceType = "index",
          Resource = [
            "index/${var.collection_name}/*"
          ],
          Permission = [
            "aoss:*"
          ]
        },
        {
          ResourceType = "assortment",
          Useful resource = [
            "collection/${var.collection_name}"
          ],
          Permission = [
            "aoss:*"
          ]
        }
      ],
      Principal = [
        data.aws_caller_identity.current.arn
      ]
    }
  ])
}

This knowledge entry coverage permits the present AWS position or consumer to carry out collection-related actions on the gathering named example-collection and index-related actions on the indexes within the assortment.

Deploy utilizing Terraform

Now that you’ve got configured the mandatory assets, apply the configuration utilizing terraform apply. Earlier than creating the assets, Terraform will describe all of the assets that will probably be created so you’ll be able to confirm your configuration:

$ terraform apply

...

Terraform used the chosen suppliers to generate the next execution plan. Useful resource actions are indicated with the next symbols:
  + create

Terraform will carry out the next actions:

...

Plan: 13 so as to add, 0 to alter, 0 to destroy.

Adjustments to Outputs:
  + collection_enpdoint = (identified after apply)
  + dashboard_endpoint  = (identified after apply)

Do you wish to carry out these actions?
  Terraform will carry out the actions described above.
  Solely 'sure' will probably be accepted to approve.

  Enter a price:

If that is the primary OpenSearch Serverless assortment in your account, making use of the configuration could take over 10 minutes as a result of Terraform waits for the gathering to change into lively.

Apply full! Assets: 13 added, 0 modified, 0 destroyed.

Outputs:

collection_enpdoint = "..."
dashboard_endpoint = "..."

You could have now deployed an OpenSearch Serverless time collection assortment with insurance policies to configure encryption and entry to the gathering!

Clear up

The assets will incur prices so long as they’re working, so clear up the assets when you find yourself performed utilizing them. Use the terraform destroy command to do that:

$ terraform destroy

...

Terraform used the chosen suppliers to generate the next execution plan. Useful resource actions are indicated with the next symbols:
  - destroy

Terraform will carry out the next actions:

  ...

Plan: 0 so as to add, 0 to alter, 13 to destroy.

Adjustments to Outputs:

...

Do you actually wish to destroy all assets?
  Terraform will destroy all of your managed infrastructure, as proven above.
  There isn't any undo. Solely 'sure' will probably be accepted to substantiate.

  Enter a price:

Reply sure to run this plan and destroy the infrastructure.

Destroy full! Assets: 13 destroyed.

All assets created throughout this walkthrough have now been deleted.

Conclusion

On this submit, you created an OpenSearch Serverless assortment. Utilizing IaC software program like Terraform could make it easy to handle your assets like OpenSearch Serverless collections, encryption, community, and knowledge entry insurance policies, and VPC endpoints.

Thanks to all of the open-source contributors who assist preserve OpenSearch and Terraform.

Attempt utilizing OpenSearch Serverless with Terraform to simplify your useful resource administration. Take a look at the Getting began with Amazon OpenSearch Serverless workshop and the Amazon OpenSearch Serverless Developer Information to be taught extra about OpenSearch Serverless.


Concerning the authors

Joshua Luo is a Software program Improvement Engineer for Amazon OpenSearch Serverless. He works on the techniques that allow prospects to handle and monitor their OpenSearch Serverless assets. He enjoys bouldering, images, and videography in his free time.

Satish Nandi is a Senior Technical Product Supervisor for Amazon OpenSearch Service.

Related Articles

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Latest Articles