TerraformでSNSトピックとAPIGateway+LambdaとS3Bucketを一撃で作成してみた
以前にTerraformで3大クラウドの仮想ネットワークを作って、その後Teraform熱は冷めていた感じだったのですが、急遽AWSの開発環境が必要になって本番環境のやつをコピーしようと思ったんですが、そこでAPIGatewayを一々ちまちまとやるのがめんどくさく、かといってServelessFrameworkでやるのもなぁと思ったのでここで登場のTerraformです。当然ながらAPIGatewayどころかLambdaとかS3とか主要どころのAWSのリソースはいじくれます。なんかCloudFormationもいじくれるらしいです。頼もしい限りです。
そこでコピペの限りを尽くして本番環境のAPIGatewayとかLambdaをバコンと自分で用意したAWSのアカウントに投入します。
とりあえず今回の準備に必要となる「SNSトピック」と「APIGateway」それと「Lambda」そして「S3」のBucketを準備します。コマンド一発で出来上がる様にしたいところですがSNSについてはメールのサブスクライバは承認が必要なのでTerraformが対応していないのでトピックのみ作成です。
・エクスポートタブを選択してSwagger+APIGateway拡張の形式を選択
・YAMLを選ぶと下にソースが表示されるのでそれをコピーして貼り付ける
もしくはyamlファイルが保管できるのでそれをアップロードする。
これだけで3日位潰れました。ほんと自分の能力のなさが嫌になります。
こういうところ調査スキルが低い上にプログラムもヨワヨワなのでもっと早く解決できるように精進したいのですが、どうすれば精進できるのか、ご教示いただけると幸いです。
https://dev.classmethod.jp/articles/s3-event-notifications-sns-sqs-by-terraform/
https://qiita.com/ritukiii/items/a2dfac9de35578543d70
https://nishipy.com/archives/1204
https://recruit-tech.co.jp/blog/2018/11/20/iac_terraform/
http://blog.father.gedow.net/2016/07/21/aws-is-too-complex/
https://www.terraform.io/docs/providers/aws/r/sns_topic.html
https://buildersbox.corp-sansan.com/entry/2020/01/15/110000
https://dev.classmethod.jp/articles/directory-layout-bestpractice-in-terraform/
https://dev.classmethod.jp/articles/ami-and-snapshot-delete-with-apex-and-terraform/
https://qiita.com/Pampus/items/038004afb606952e7cd6
https://qiita.com/ringo/items/3af1735cd833fb80da75
https://qiita.com/fukushi_yoshikazu/items/e68ca839e0a56152ab85
https://qiita.com/minamijoyo/items/1f57c62bed781ab8f4d7
https://blog.d-shimizu.io/article/1371
http://micpsm.hatenablog.com/entry/2019/12/08/120000
https://github.com/spring-media/terraform-aws-lambda/blob/master/outputs.tf
https://github.com/hashicorp/terraform/issues/21304
https://learn.hashicorp.com/terraform/aws/lambda-api-gateway
https://qiita.com/CkReal/items/be0923f6352b0109e225
https://qiita.com/tsukapah/items/8e0cf03aff49a3c565f8
https://c4se.hatenablog.com/entry/2016/07/26/122508
https://www.beex-inc.com/blog/article-4198/
https://www.vic-l.com/aws-lamba-and-api-gateway-integration-with-terraform-to-collect-emails/
http://quabr.com:8182/59142560/terraform-api-gateway-with-lambda-integration
https://stackoverflow.com/questions/59142560/terraform-api-gateway-with-lambda-integration
https://dev.to/rolfstreefkerk/openapi-with-terraform-on-aws-api-gateway-17je
https://www.korken.xyz/posts/aws-lambda/
https://charlieegan3.com/posts/2017-08-10-terraform-lambda-api-gateway/
https://www.rockedscience.net/articles/api-gateway-logging-with-terraform/
https://www.endava.com/en/blog/Engineering/2019/AWS-serverless-with-Terraform
https://qiita.com/tksugimoto/items/33f9fe6aa48a1343e360
https://krrrr.hatenablog.com/entry/2017/02/01/213938
https://medium.com/onfido-tech/aws-api-gateway-with-terraform-7a2bebe8b68f
https://ksoichiro.blogspot.com/2020/01/terraform-api-gatewaydeploymentstagesta.html
https://registry.terraform.io/modules/clouddrove/api-gateway/aws/0.12.1
https://buginfo.tech/questions-943041.htm
https://sysgears.com/articles/moving-lambda-function-from-serverless-to-terraform/
https://labs.septeni.co.jp/entry/2019/12/13/120000
https://www.slideshare.net/sachirouinoue/serverless-framework
https://johncodeinaire.com/aws-cognito-terraform-tutorial/
https://future-architect.github.io/articles/20190903/
https://qiita.com/ymmy02/items/e7368abd8e3dafbc5c52
https://github.com/terraform-providers/terraform-provider-aws/issues/4872
https://stackoverflow.com/questions/59715228/terraform-api-gateway-integration-with-swagger-localstack
https://www.reddit.com/r/Terraform/comments/8jos55/terraform_for_lambdaapi_gateway_using_swagger/
そこでコピペの限りを尽くして本番環境のAPIGatewayとかLambdaをバコンと自分で用意したAWSのアカウントに投入します。
とりあえず今回の準備に必要となる「SNSトピック」と「APIGateway」それと「Lambda」そして「S3」のBucketを準備します。コマンド一発で出来上がる様にしたいところですがSNSについてはメールのサブスクライバは承認が必要なのでTerraformが対応していないのでトピックのみ作成です。
◆設定ファイルやコマンド関連など
#事前準備(環境変数に埋込) $ export TF_VAR_access_key="アクセスキー" $ export TF_VAR_secret_key="シークレットキー" $ export TF_VAR_role_arn="IAMロールのARN" #1.作業用ディレクトリ作成 $ mkdir apigwsample $ cd apigwsample $ mkdir apigateway lambda s3 #2.ルートのtfファイル作成 $ vi main.tf ※以下の内容を新規作成 --------------------- resource "aws_sns_topic" "this_sns_topic" { name = var.sns_topic_name } module "lambda" { source = "./lambda" } module "apigateway" { source = "./apigateway" region = var.region account_id = var.account_id function_name = "${module.lambda.function_name}" } module "s3" { source = "./s3" } --------------------- $ vi provider.tf ※以下の内容を新規作成 --------------------- provider "aws" { access_key = var.access_key secret_key = var.secret_key region = var.region } --------------------- $ vi variables.tf ※以下の内容を新規作成 --------------------- variable "access_key" { description = "AWS Access Key" } variable "secret_key" { description = "AWS Secret Key" } variable "role_arn" { description = "AWS Role ARN for Assume Role" } variable "region" { description = "AWS Region" default = "ap-northeast-1" } variable "sns_topic_name" { default = "sample_topic" } variable "account_id" { default = "XXXXXXXXXXXX" } --------------------- $ terraform fmt #3.Lambdaのtfファイル作成 $ cd lambda $ vi main.tf ※以下の内容を新規作成 --------------------- data "archive_file" "arc_dev_function" { type = "zip" source_dir = "${path.module}/src/" output_path = "${path.module}/upload/lambda_functions.zip" } resource "aws_lambda_function" "nlife_dev_function" { filename = data.archive_file.arc_dev_function.output_path function_name = var.function_name role = var.lambda_function_arn handler = "lambda_function.lambda_handler" source_code_hash = data.archive_file.arc_dev_function.output_base64sha256 runtime = var.lambda_runtime memory_size = 128 timeout = 30 environment { variables = { "SAMPLE_ENV" = var.environment_value_1 } } tags = { "CreateOwner" = var.tag_value_1 } } --------------------- $ vi variables.tf ※以下の内容を新規作成 --------------------- variable "lambda_role_arn" { default = "AWS Role ARN for Assume Role" } variable "lambda_runtime" { default = "python3.8" } variable "function_name" { default = "tr_sample_function" } variable "lambda_function_arn" { default = "arn:aws:iam::account-id:role/rolename" #←あらかじめマネジメントコンソールでロール作成 } variable "environment_value_1" { default = "sample_env" } variable "tag_value_1" { default = "Terraform" } --------------------- $ vi outputs.tf ※以下の内容を新規作成 --------------------- output "function_name" { value = aws_lambda_function.dev_function.id } --------------------- $ mkdir src upload $ vi src/lambda_function.py ※以下の内容を新規作成 --------------------- import json # -*- coding:utf-8 -*- from urllib.parse import parse_qs def lambda_handler(event, context): try: user_object =[ { 'id' : 12345, 'name':'山下 清', 'age' : 30, 'url': 'プロフィール' }, { 'id' : 22222, 'name':'上田 尊司', 'age' : 45, 'url': 'プロフィール' }, { 'id' : 55555, 'name':'清宮 仁郎', 'age' : 28, 'url': 'プロフィール' }, { 'id' : 69156, 'name':'ヘンリー 仁郎', 'age' : 35, 'url': 'プロフィール' } ] return user_object except Exception as e: print(e) raise e --------------------- $ terraform fmt $ cd ../ #4.apigatewayのtfファイル作成 $ cd apigateway $ vi main.tf ※以下の内容を新規作成 --------------------- data "template_file" "apigateway_template" { template = file("${path.module}/swagger_definition.yml") vars = { region = var.region account_id = var.account_id function_name = var.function_name base_path = var.apigateway_stg_name } } resource "aws_api_gateway_rest_api" "my_api_gateway" { name = var.function_name description = "This is my API for demonstration purposes" body = data.template_file.apigateway_template.rendered } resource "aws_api_gateway_deployment" "apiDeployment" { rest_api_id = aws_api_gateway_rest_api.my_api_gateway.id stage_name = var.apigateway_stg_name } resource "aws_lambda_permission" "lambda_permission" { statement_id = "AllowMyDemoAPIInvoke" action = "lambda:InvokeFunction" function_name = var.function_name principal = "apigateway.amazonaws.com" source_arn = "${aws_api_gateway_rest_api.my_api_gateway.execution_arn}/*/*/*" } --------------------- $ vi variables.tf ※以下の内容を新規作成 --------------------- variable "apigateway_stg_name" { default = "prod" } variable "region" {} variable "account_id" {} variable "function_name" {} --------------------- $ vi swagger_definition.yml ※設定済みのAPIGatewayのyamlをエクスポートして記述変更 --------------------- swagger: "2.0" info: description: "Created by terraform" version: "2020-05-14T03:30:40Z" title: "Terraform-sample" host: "XXXX.execute-api.ap-northeast-1.amazonaws.com" basePath: "${base_path}" schemes: - "https" paths: /regist: get: produces: - "application/json" responses: 200: description: "200 response" schema: $ref: "#/definitions/Empty" headers: Access-Control-Allow-Origin: type: "string" x-amazon-apigateway-integration: uri: "arn:aws:apigateway:${region}:lambda:path/2015-03-31/functions/arn:aws:lambda:${region}:${account_id}:function:${function_name}/invocations" responses: default: statusCode: "200" responseParameters: method.response.header.Access-Control-Allow-Origin: "'*'" passthroughBehavior: "when_no_match" httpMethod: "POST" contentHandling: "CONVERT_TO_TEXT" type: "aws" options: consumes: - "application/json" produces: - "application/json" responses: 200: description: "200 response" schema: $ref: "#/definitions/Empty" headers: Access-Control-Allow-Origin: type: "string" Access-Control-Allow-Methods: type: "string" Access-Control-Allow-Headers: type: "string" x-amazon-apigateway-integration: responses: default: statusCode: "200" responseParameters: method.response.header.Access-Control-Allow-Methods: "'GET,OPTIONS'" method.response.header.Access-Control-Allow-Headers: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'" method.response.header.Access-Control-Allow-Origin: "'*'" requestTemplates: application/json: "{\"statusCode\": 200}" passthroughBehavior: "when_no_match" type: "mock" definitions: Empty: type: "object" title: "Empty Schema" --------------------- $ terraform fmt $ cd ../ #5.S3のtfファイル作成 $ cd s3 $ vi main.tf ※以下の内容を新規作成 --------------------- resource "aws_s3_bucket" "b" { bucket = var.bucket_name acl = "private" tags = { Name = "My bucket" Environment = "Dev" } } --------------------- $ vi variables.tf ※以下の内容を新規作成 --------------------- variable "bucket_name" { default = "XXXXXXX" #他とかぶらない一意な名前で } --------------------- $ terraform fmt $ cd ../ #6.デプロイ $ terraform init $ terraform plan $ terraform apply ~中略~ Enter a value: yes #7.後片付け $ terraform destroy #8.環境など $ terraform --version Terraform v0.12.24 + provider.archive v1.3.0 + provider.aws v2.61.0 + provider.template v2.1.2 $ tree --charset=o . |-- apigateway | |-- main.tf | |-- swagger_definition.yml | `-- variables.tf |-- lambda | |-- main.tf | |-- outputs.tf | |-- src | | `-- lambda_function.py | |-- upload | | `-- lambda_functions.zip | `-- variables.tf |-- main.tf |-- provider.tf |-- s3 | |-- main.tf | `-- variables.tf |-- terraform.tfstate |-- terraform.tfstate.backup `-- variables.tf
◆AWSのAPIGatewayのyamlエクスポート方法
・ステージの箇所でルートのステージを選択する。・エクスポートタブを選択してSwagger+APIGateway拡張の形式を選択
・YAMLを選ぶと下にソースが表示されるのでそれをコピーして貼り付ける
もしくはyamlファイルが保管できるのでそれをアップロードする。
これだけで3日位潰れました。ほんと自分の能力のなさが嫌になります。
こういうところ調査スキルが低い上にプログラムもヨワヨワなのでもっと早く解決できるように精進したいのですが、どうすれば精進できるのか、ご教示いただけると幸いです。
◆参考サイト
・SNSトピック関連
https://dev.classmethod.jp/articles/amazon-sns-2017/https://dev.classmethod.jp/articles/s3-event-notifications-sns-sqs-by-terraform/
https://qiita.com/ritukiii/items/a2dfac9de35578543d70
https://nishipy.com/archives/1204
https://recruit-tech.co.jp/blog/2018/11/20/iac_terraform/
http://blog.father.gedow.net/2016/07/21/aws-is-too-complex/
https://www.terraform.io/docs/providers/aws/r/sns_topic.html
https://buildersbox.corp-sansan.com/entry/2020/01/15/110000
https://dev.classmethod.jp/articles/directory-layout-bestpractice-in-terraform/
・Lambda関連
https://dev.classmethod.jp/articles/terraform-lambda-deployment/https://dev.classmethod.jp/articles/ami-and-snapshot-delete-with-apex-and-terraform/
https://qiita.com/Pampus/items/038004afb606952e7cd6
https://qiita.com/ringo/items/3af1735cd833fb80da75
https://qiita.com/fukushi_yoshikazu/items/e68ca839e0a56152ab85
https://qiita.com/minamijoyo/items/1f57c62bed781ab8f4d7
https://blog.d-shimizu.io/article/1371
http://micpsm.hatenablog.com/entry/2019/12/08/120000
https://github.com/spring-media/terraform-aws-lambda/blob/master/outputs.tf
・ZIPアーカイブできないトラブル
https://stackoverflow.com/questions/57054136/how-to-resolve-this-error-archiving-directory-could-not-archive-missing-directhttps://github.com/hashicorp/terraform/issues/21304
・APIGateway関連
https://www.terraform.io/docs/providers/aws/r/api_gateway_rest_api.htmlhttps://learn.hashicorp.com/terraform/aws/lambda-api-gateway
https://qiita.com/CkReal/items/be0923f6352b0109e225
https://qiita.com/tsukapah/items/8e0cf03aff49a3c565f8
https://c4se.hatenablog.com/entry/2016/07/26/122508
https://www.beex-inc.com/blog/article-4198/
https://www.vic-l.com/aws-lamba-and-api-gateway-integration-with-terraform-to-collect-emails/
http://quabr.com:8182/59142560/terraform-api-gateway-with-lambda-integration
https://stackoverflow.com/questions/59142560/terraform-api-gateway-with-lambda-integration
https://dev.to/rolfstreefkerk/openapi-with-terraform-on-aws-api-gateway-17je
https://www.korken.xyz/posts/aws-lambda/
https://charlieegan3.com/posts/2017-08-10-terraform-lambda-api-gateway/
https://www.rockedscience.net/articles/api-gateway-logging-with-terraform/
https://www.endava.com/en/blog/Engineering/2019/AWS-serverless-with-Terraform
https://qiita.com/tksugimoto/items/33f9fe6aa48a1343e360
https://krrrr.hatenablog.com/entry/2017/02/01/213938
https://medium.com/onfido-tech/aws-api-gateway-with-terraform-7a2bebe8b68f
https://ksoichiro.blogspot.com/2020/01/terraform-api-gatewaydeploymentstagesta.html
https://registry.terraform.io/modules/clouddrove/api-gateway/aws/0.12.1
https://buginfo.tech/questions-943041.htm
https://sysgears.com/articles/moving-lambda-function-from-serverless-to-terraform/
https://labs.septeni.co.jp/entry/2019/12/13/120000
https://www.slideshare.net/sachirouinoue/serverless-framework
https://johncodeinaire.com/aws-cognito-terraform-tutorial/
https://future-architect.github.io/articles/20190903/
https://qiita.com/ymmy02/items/e7368abd8e3dafbc5c52
・Swager取り込む方法
https://stackoverflow.com/questions/54047171/deploy-api-gateway-with-terraform-based-on-a-swagger-filehttps://github.com/terraform-providers/terraform-provider-aws/issues/4872
https://stackoverflow.com/questions/59715228/terraform-api-gateway-integration-with-swagger-localstack
https://www.reddit.com/r/Terraform/comments/8jos55/terraform_for_lambdaapi_gateway_using_swagger/
コメント