GCPの静的サイトホスティングをCloudDeploymentManagerで構築してみる

前回はAWSでしたが、今回はGCPにチャレンジしてみました。CloudDeploymentManagerLBを使った静的サイトホスティングの構成を構築できるようにしてみようというやつです。

日本語で同じようなことやってるサイトがあまりなく、英語サイトからの丸パクリです。ツヨツヨな人でないとGoogleはハードルが高いです。公式ドキュメント読んでもさっぱりだし、自分のダメさ加減を痛感します。

相変わらずへっぽこですが、何かの時に使うかもしれないので備忘録でメモとしておいておこうかと思い公開した次第です。ご指摘とかアドバイスあれば頂戴できると幸いです。2021年2月20日時点の情報です。

・環境

Google Cloud SDK 327.0.0
alpha 2021.02.05
app-engine-python 1.9.91
beta 2021.02.05
bq 2.0.64
cloud-datastore-emulator 2.1.0
core 2021.02.05
gsutil 4.58
Linux/5.4.0-65-generic
Python 3.7.7


◆作成したもの

1.ディレクトリ構造

|-- create_stack.sh
|-- delete_stack.sh
`-- yaml
    |-- backendbuckets-template.jinja
    |-- dns-template.jinja
    |-- httpproxies-template.jinja
    |-- ipaddress-template.jinja
    |-- loadbalancer-template.jinja
    |-- ssl-template.jinja
    |-- static-site-create.yaml
    `-- storage-template.jinja


2.シェルスクリプト群

スタック構築のテンプレートを呼び出すShellスクリプト

・create_stack.sh

#!/bin/bash
cd `dirname $0`
SLEEP_TIME=30
HTTP_STATUS_CODE=200

if [ $# != 1 ]; then
    echo "Empty Domain! Please [./create_stacks.sh example.com]"
    exit 1
fi
domain=$1
sitename=${domain/./-}

cp yaml/static-site-create.yaml yaml/${sitename}-static-site-create.yaml
sed -i -e "s/SITENAME/${sitename}/g" -e "s/DOMAIN/${domain}/g" yaml/${sitename}-static-site-create.yaml



echo "*** Deploymentmanager Run ***"
gcloud deployment-manager deployments create ${sitename}-static-site-deployment --config yaml/${sitename}-static-site-create.yaml

echo "*** dnssec on ***"
gcloud dns managed-zones update  ${sitename}-dnszone --dnssec-state on

echo "*** Web Sit Setting ***"
gsutil rsync -R ~/html/dist/ gs://site-${sitename}
gsutil iam ch allUsers:objectViewer gs://site-${sitename}
gsutil web set -m index.html -e 404.html gs://site-${sitename}

echo "*** Please Setting Your Domain Nameserver ***"
ns_records=$(gcloud dns record-sets list \
        --zone="${sitename}-dnszone" \
        --name="${domain}" \
        --type="NS" \
        | grep -oP "ns-cloud-[0-9a-zA-Z?=#+_&:/.%]+")
array=(${ns_records//$'\n'/ })
for i in ${array[@]}
do
  echo ${i}
done
echo "*** http statuscode check ***"
while true
do
   echo "Check now please wait・・・"
   sleep ${SLEEP_TIME}
   naked_code=$(echo $(curl -s https://${domain} -o /dev/null -w "%{http_code}\n"))
   www_code=$(echo $(curl -s https://www.${domain} -o /dev/null -w "%{http_code}\n"))
   if [ "$naked_code" -eq "$HTTP_STATUS_CODE" ] && [ "$www_code" -eq "$HTTP_STATUS_CODE" ]; then
      break
   fi
done
echo "*** Setting Complete ***"


スタックを削除するShellスクリプト

・delete_stack.sh

#!/bin/bash

cd `dirname $0`

if [ $# != 1 ]; then
    echo 'Empty Domain! Please [./deletestack.sh example.com]'
    exit 1
fi
domain=$1
sitename=${domain/./-}

gsutil rm gs://site-${sitename}/**

echo '*** Delete Stack ***'
gcloud deployment-manager deployments delete ${sitename}-static-site-deployment
rm yaml/${sitename}-static-site-create.yaml


3.DeploymentManagerテンプレート

全体のテンプレートをまとめて実行するテンプレートです

・static-site-create.yaml

imports:
 - path: ipaddress-template.jinja
 - path: dns-template.jinja
 - path: ssl-template.jinja
 - path: storage-template.jinja
 - path: backendbuckets-template.jinja
 - path: loadbalancer-template.jinja
 - path: httpproxies-template.jinja

resources:
 - name: ipaddress
   type: ipaddress-template.jinja
   properties:
     site-name: SITENAME
 - name: dnsrecords
   type: dns-template.jinja
   properties:
     site-name: SITENAME
     site-domain: DOMAIN
     static-ip-v4-addr: $(ref.ipaddress.ipv4)
     static-ip-v6-addr: $(ref.ipaddress.ipv6)
 - name: sslcertificates
   type: ssl-template.jinja
   properties:
     site-name: SITENAME
     site-domain: DOMAIN
 - name: storagebucket
   type: storage-template.jinja
   properties:
     site-name: SITENAME
 - name: backendbuckets
   type: backendbuckets-template.jinja
   properties:
     site-name: SITENAME
 - name: loadbalancer
   type: loadbalancer-template.jinja
   properties:
     backend-buckets: $(ref.backendbuckets.backend)
     site-name: SITENAME
 - name: httpproxies
   type: httpproxies-template.jinja
   properties:
     site-name: SITENAME
     ssl-certificate: $(ref.sslcertificates.cert)
     http-urlmap: $(ref.loadbalancer.http)
     https-urlmap: $(ref.loadbalancer.https)
     ipv4: $(ref.ipaddress.ipv4)
     ipv6: $(ref.ipaddress.ipv6)


固定IP取得用テンプレートです(IP-V4,V6両方を取得)

・ipaddress-template.jinja

resources:
  - name: {{ properties["site-name"] }}-ipv4
    type: compute.v1.globalAddress
    properties:
      ipVersion: IPV4
  - name: {{ properties["site-name"] }}-ipv6
    type: compute.v1.globalAddress
    properties:
      ipVersion: IPV6
outputs:
  - name: ipv4
    value: $(ref.{{ properties["site-name"] }}-ipv4.address)
  - name: ipv6
    value: $(ref.{{ properties["site-name"] }}-ipv6.address)


DNSゾーンを作成しAとAAAAレコードを登録するテンプレート

・dns-template.jinja

resources:
  - name:  {{ properties["site-name"] }}-dnszone
    type: gcp-types/dns-v1:managedZones
    properties:
      description: StaticSiteSample
      dnsName:  {{ properties["site-domain"] }}.
      dnssecConfig.state: on
  - name: {{ properties["site-name"] }}-naked-a-record
    type: gcp-types/dns-v1:resourceRecordSets
    properties:
      name: tohonokai.cf.
      managedZone: $(ref.{{ properties["site-name"] }}-dnszone.name)
      records:
      - name: {{ properties["site-domain"] }}.
        type: A
        ttl: 300
        rrdatas:
        - {{ properties["static-ip-v4-addr"] }}
  - name: {{ properties["site-name"] }}-www-a-record
    type: gcp-types/dns-v1:resourceRecordSets
    properties:
      name: www.{{ properties["site-domain"] }}.
      managedZone: $(ref.{{ properties["site-name"] }}-dnszone.name)
      records:
      - type: A
        ttl: 300
        rrdatas:
        - {{ properties["static-ip-v4-addr"] }}
  - name: {{ properties["site-name"] }}-naked-aaaa-record
    type: gcp-types/dns-v1:resourceRecordSets
    properties:
      name: tohonokai.cf.
      managedZone: $(ref.{{ properties["site-name"] }}-dnszone.name)
      records:
      - name: {{ properties["site-domain"] }}.
        type: AAAA
        ttl: 300
        rrdatas:
        - {{ properties["static-ip-v6-addr"] }}
  - name: {{ properties["site-name"] }}-www-aaaa-record
    type: gcp-types/dns-v1:resourceRecordSets
    properties:
      name: www.{{ properties["site-domain"] }}.
      managedZone: $(ref.{{ properties["site-name"] }}-dnszone.name)
      records:
      - type: AAAA
        ttl: 300
        rrdatas:
        - {{ properties["static-ip-v6-addr"] }}


GoogleマネージドSSL発行用のテンプレート

・ssl-template.jinja

resources:
  - name: {{ properties["site-name"] }}-ssl
    type: compute.v1.sslCertificate
    properties:
      type: MANAGED
      managed:
        domains:
        - {{ properties["site-domain"] }}
        - www.{{ properties["site-domain"] }}
outputs:
  - name: cert
    value: $(ref.{{ properties["site-name"] }}-ssl.selfLink)


静的ホスティング用とログ用のストレージ作成用テンプレート

・storage-template.jinja

resources:
  - name: log-{{ properties["site-name"] }}
    type: gcp-types/storage-v1:buckets
    properties:
      predefinedAcl: projectPrivate
      projection: full
      location: asia-east2
      storageClass: REGIONAL
  - name: site-{{ properties["site-name"] }}
    type: gcp-types/storage-v1:buckets
    properties:
      location: asia
      locationType: multi-region
      defaultObjectAcl:
      - bucket: {{ properties["site-name"] }}
        entity: allUsers
        role: READER
      website:
        mainPageSuffix: index.html
        notFoundPage : 404.html
      logging:
        logBucket: $(ref.log-{{ properties["site-name"] }}.name)
        logObjectPrefix: {{ properties["site-name"] }}
outputs:
  - name: sitebucket
    value: $(ref.site-{{ properties["site-name"] }}.name)
  - name: logbucket
    value: $(ref.log-{{ properties["site-name"] }}.name)

バックエンドバケット作成用テンプレート

・backendbuckets-template.jinja

resources:
  - name: {{ properties["site-name"] }}-backend
    type: compute.beta.backendBucket
    properties:
      bucketName: $(ref.site-{{ properties["site-name"] }}.name)
outputs:
  - name: backend
    value: $(ref.{{ properties["site-name"] }}-backend.selfLink)


URLマップ作成用テンプレート

・loadbalancer-template.jinja

resources:
  - name: {{ properties["site-name"] }}-http-urlmap
    type: compute.v1.urlMap
    properties:
      defaultUrlRedirect:
        httpsRedirect: true
        redirectResponseCode: MOVED_PERMANENTLY_DEFAULT
  - name: {{ properties["site-name"] }}-https-urlmap
    type: compute.v1.urlMap
    properties:
      defaultService: {{ properties["backend-buckets"] }}
      hostRules:
      - hosts: ["*"]
        pathMatcher: "allpaths"
      pathMatchers:
      - name: "allpaths"
        defaultService: {{ properties["backend-buckets"] }}
        pathRules:
        - service: {{ properties["backend-buckets"] }}
          paths: ["/*"]
outputs:
  - name: http
    value: $(ref.{{ properties["site-name"] }}-http-urlmap.selfLink)
  - name: https
    value: $(ref.{{ properties["site-name"] }}-https-urlmap.selfLink)


httpプロキシ作成用テンプレート

・httpproxies-template.jinja

resources:
  - name: {{ properties["site-name"] }}-http-proxy
    type: compute.v1.targetHttpProxy
    properties:
      urlMap: {{ properties["http-urlmap"] }}
  - name: {{ properties["site-name"] }}-https-proxy
    type: compute.v1.targetHttpsProxy
    properties:
      urlMap: {{ properties["https-urlmap"] }}
      sslCertificates:
        - {{ properties["ssl-certificate"] }}
  - name: lb-{{ properties["site-name"] }}-http-v4-forwardingrule
    type: compute.v1.globalForwardingRule
    properties:
      target: $(ref.{{ properties["site-name"] }}-http-proxy.selfLink)
      IPAddress: {{ properties["ipv4"] }}
      IPProtocol: TCP
      portRange: 80-80
  - name: lb-{{ properties["site-name"] }}-http-v6-forwardingrule
    type: compute.v1.globalForwardingRule
    properties:
      target: $(ref.{{ properties["site-name"] }}-http-proxy.selfLink)
      IPAddress: {{ properties["ipv6"] }}
      IPProtocol: TCP
      portRange: 80-80
  - name: lb-{{ properties["site-name"] }}-https-v4-forwardingrule
    type: compute.v1.globalForwardingRule
    properties:
      target: $(ref.{{ properties["site-name"] }}-https-proxy.selfLink)
      IPAddress: {{ properties["ipv4"] }}
      IPProtocol: TCP
      portRange: 443-443
  - name: lb-{{ properties["site-name"] }}-https-v6-forwardingrule
    type: compute.v1.globalForwardingRule
    properties:
      target: $(ref.{{ properties["site-name"] }}-https-proxy.selfLink)
      IPAddress: {{ properties["ipv6"] }}
      IPProtocol: TCP
      portRange: 443-443


DNSの伝播がとんでもなく遅いのであまり意味ないかもしれないけどCloudDNSSOAレコードNSレコードTTLを300秒程度に手動で変更してたりしています。まぁ意味ないんでしょうけど・・・

そしてDNSとかSSLが絡むと自動化の難易度が一気に上がって辛くなります。いよいよ本丸のTerraformかな・・・。


◆参考サイト

・コピペ元(w

https://stackoverflow.com/questions/53648159/how-to-redirect-http-to-https-using-gcp-load-balancer?fbclid=IwAR0hYww_uZ2si24Zzu-_GXdtoBkLbPTAes8pgLESke-0MpxlVTlcvbPBKtQ

・CloudDeploymentManager

https://cloud.google.com/deployment-manager/docs/quickstart?hl=ja

https://qiita.com/shotaTsuge/items/bb995f4e2f3f6ae9d12d

https://future-architect.github.io/articles/20200219/

https://cloud.google.com/deployment-manager/docs

https://github.com/GoogleCloudPlatform/deploymentmanager-samples/tree/master/templates

https://qiita.com/shojimotio/items/f303007e9d9b900d5b52

https://dev.classmethod.jp/articles/gcp-cloud-deployment-manager-cloud-functions/

https://cloud.google.com/deployment-manager/docs/configuration/supported-resource-types?hl=ja

https://cloud.google.com/deployment-manager/docs/configuration/use-references?hl=ja

https://cloud.google.com/deployment-manager/docs/configuration/syntax-reference?hl=ja

https://cloud.google.com/deployment-manager/docs/configuration/expose-information-outputs?hl=ja

https://github.com/GoogleCloudPlatform/deploymentmanager-samples/tree/master/google/resource-snippets

https://cloud.google.com/deployment-manager/docs/at-scale?hl=ja

https://pc.atsuhiro-me.net/entry/2021/01/04/200658

https://this.aereal.org/entry/2018/08/21/192317

・サンプルのテンプレート

https://github.com/GoogleCloudPlatform/cloud-foundation-toolkit/tree/master/dm/templates/ip_reservation

https://github.com/GoogleCloudPlatform/cloud-foundation-toolkit/tree/master/dm/templates/ssl_certificate

https://github.com/GoogleCloudPlatform/cloud-foundation-toolkit/tree/master/dm/templates/gcs_bucket

https://github.com/GoogleCloudPlatform/cloud-foundation-toolkit/tree/master/dm/templates/external_load_balancer

・外部IPアドレス予約

https://saba.omnioo.com/note/6112/gcloud%E3%81%A7%E9%9D%99%E7%9A%84%E5%A4%96%E9%83%A8ip%E3%82%A2%E3%83%89%E3%83%AC%E3%82%B9%E3%80%81%E9%9D%99%E7%9A%84%E5%86%85%E9%83%A8ip%E3%82%A2%E3%83%89%E3%83%AC%E3%82%B9%E3%82%92%E4%BA%88%E7%B4%84/

https://cloud.google.com/sdk/gcloud/reference/compute/addresses/create

https://access.redhat.com/documentation/ja-jp/openshift_container_platform/4.5/html/installing_on_gcp/installing-gcp-user-infra-vpc

https://cloud.google.com/compute/docs/reference/rest/v1/addresses?hl=ja

https://stackoverflow.com/questions/62659711/google-deployment-manager-error-when-using-manual-ip-allocation-in-nat-http-400

https://cloud.google.com/compute/docs/ip-addresses/reserve-static-external-ip-address?hl=ja

・CloudDNS

https://yukidarake.hateblo.jp/entry/2016/04/26/203318

https://github.com/GoogleCloudPlatform/deploymentmanager-samples/blob/master/google/resource-snippets/dns-v1/managed-zone.yaml

https://cloud.google.com/dns/docs/records?hl=ja#gcloud

https://stackoverflow.com/questions/60303394/google-deployment-manager-dns-removing-txt-record-also-removes-a-record

https://plugaru.org/2018/08/21/managing-dns-zones-with-gcp-deployment-manager/

・ロードバランサー

https://cloud.google.com/load-balancing/docs/ipv6

https://codeblog.dotsandbrackets.com/external-load-balancer/

https://stackoverflow.com/questions/54959122/ssl-issues-with-jinja-https-load-balancer-google-cloud

https://stackoverflow.com/questions/58920827/gcp-cloud-deployment-load-balancer-backend-bucket

https://ja.ojit.com/so/google-cloud-platform/3835319

https://medium.com/google-cloud/infrastructure-as-code-on-google-cloud-platform-load-balancers-22591561a2ec

https://yamavlog.com/gcp-loadblancer-redirect-https/

・GCSの作成

https://cloud.google.com/storage/docs/json_api/v1/buckets?hl=ja

https://dev.classmethod.jp/articles/gs-deployment-manager-gcs-bigquery-create/

https://googleapis.github.io/google-cloud-dotnet/docs/Google.Cloud.Storage.V1/api/Google.Cloud.Storage.V1.StorageClient.html

https://zenn.dev/nekoshita/articles/5730d79bc39a20

https://cloud.google.com/storage/docs/access-logs?hl=ja

https://hawksnowlog.blogspot.com/2019/02/cloud-storage-enable-logging.html#%E3%83%90%E3%82%B1%E3%83%83%E3%83%88%E3%81%AE%E3%83%AD%E3%82%AE%E3%83%B3%E3%82%B0%E3%82%92-on-%E3%81%AB%E3%81%99%E3%82%8B

https://github.com/GoogleCloudPlatform/deploymentmanager-samples/blob/master/google/resource-snippets/storage-v1/bucket.jinja

・SSL証明書

https://www.apps-gcp.com/gcp%E3%81%A7%E7%84%A1%E6%96%99https%E5%8C%96%EF%BD%9Bcompute-engine%E7%B7%A8%EF%BD%9D/

https://stackoverflow.com/questions/55376106/use-cloud-deployment-manager-to-upload-ssl-certificates

https://cloud.google.com/compute/docs/reference/rest/v1/sslCertificates

https://cloud.google.com/load-balancing/docs/ssl-certificates/google-managed-certs?hl=ja#api

https://tutorialmore.com/questions-302176.htm

https://github.com/GoogleCloudPlatform/gke-managed-certs/issues/13

・その他

https://dominicusin.github.io/2019/07/25/gcloud-cheat-sheet.html


◆GitHub

とりあえずGitHub上げときます

コメント

このブログの人気の投稿

証券外務員1種勉強(計算式暗記用メモ)

GASでGoogleDriveのサブフォルダとファイル一覧を出力する

マクロ経済学(IS-LM分析)