Overview
- CodeCommit: storing code
- CodePipeline: automating our pipeline from code to ElasticBeanstalk
- CodeBuild: building and testing our code
- CodeDeploy: deploying the code to EC2 fleets (not Beanstalk) Impportant! CodeDeploy only deploy to EC2, no whereelse
CodeCommit
Security
Authentication in Git:
- SSH Key
- HTTPS
- MFA
Authorization in Git:
- IAM policies for user / role to repositories
Encryption:
- Repo are encrypted at rest using KMS automatically
- Encrypted in transit (SSH or HTTPS)
Cross Account access:
- Use IAM role in your AWS account and use AWS STS (with AssumeRole API)
- NOT share SSH
- NOT share AWS credentials
Notifications
You can trigger notification in CodeCommit using SNS, Lambda or CloudWatch Event Rules.
Basiclly any notification related to Pull request (inlcude comments), goes to CloudWatch Event Rules.
Otherwise goes to SNS / AWS Lambda.
CloudWath Event rules can also goes into SNS topic.
SNS / Lambda:
- Deletion of branches
- Push to master
- Notify external Build System
- Trigger AWS Lambda function to perform codebase analysis (maybe credentials got committed in the code?)
CloudWatch Event Rules:
- Trigger for pull request (Created/updated/deleted/commented)
- Commit comment events
- CloudWatch Event Rules goes into a SNS topic
Answer: CloudWatch Event Rules. Anything related to PR, goes to CER.
CodePipeline
Artifacts
- Each pipeline stage can create "artifacts"
- Artifacts are passed stored in Amamzon S3 and passed on to next stage
Troubleshooting
- CodePipeline state changes happen in AWS CloudWatch Events, which can in return create SNS notifications.
- you can create events for failed piplines
- you can create events for cancelled stages
Pipeline -> all goes to CloudWatch Event Rules -> SNS
Different from CodeCommit, only Pull Request related goes into CloudWatch Event Rules
- If CodePipeline fails a stage, you pipeline stops and you can get information in the console
- AWS CloudTrail can be used to audit AWS API calls
- If Pipeline cannot perform an action, make sure the "IAM Service Role" attached does have enough permission (IAM Policy)
Deploy
Services which Codepipeline can deploy to:
Stage
You can new stage easily and each stage can have multi action groups (Sequential / parallel)
CodeBuild
- Continuous scaling
- Pay for usage
- Leverage Docker under the hood for reproducible builds
- Possibility to extend capabilities leveraging our own base Docker images
- Intergration with KMS for encryption of build artifacts
- IAM for build permissions
- VPC netowrk security
- CloudTrail for API calls logging
- Build instruction can be defined in code (buildspec.yml file)
- Output logs to S3 & AWS CloudWatch Logs
- Use CloudWatch Alarms to detect failed build and trigger notification
- CloudWatch Events / AWS Lambda as a Glue
- SNS notifications
- Ability to reproduce CodeBuild locally to troubleshoot in case of errors
- Builds can be defined within CodePipeline or CodeBuild itself
- Deployment group are set of EC2 instance where you are going to deploy to
Supported environment
- If not in lists, then you can use Docker to extend any environment you linke
buildspec.yml
- Must be at root of your code
- Define environment variables
- Plaintext variables
- Secure secreets: use SSM Parameter store
- Phases
- Install: install dependencies you may need for your build
- Pre build: final commands to execute before build
- Build: actual build commands
- Post build: finishing touches (zip output for example)
- Artifacts: upload to s3 encrypted with KMS
- Cache: Files to cache (usually dependencies) to S3 for future build speedup
CodeBuild containers are deleted at the end of their execution. You cannot SSH into them, even while they are running.
CodePipeline can deploy to S3, so need to choose CodePipeline
CodeBuild is mainly used for transform markdown file to html file in build phrase.
CodeDeploy
- Deploy application automatically to many EC2 instance
- EC2 instnaces are grouped by deployment group (dev/test/prod)
- CodeDeploy works with any application, auto scaling integration
- Blue/green only works with EC2 instances (NOT no premise)
- Support for AWS Labda
- CodeDploy doesn't previson resources
AppSpec
Hooks: set of instructions to do to deploy the new version (hooks can have timeouts).
The order is:
- BeforeBlockTraffic
- AfterBlockTraffic
- ApplicationStop
- DownloadBundle
- BeforeInstall
- AfterInstall
- ApplicationStart
- ValidateService: really important (to make sure application is indeed working)
- BeforeAllowTraffic
- AllowTraffic
- AfterAllowTraffic
Failures:
- New deployments wil first be deployed to "failed state" instance
- To rolback: redploy old deployment or enable automated rollback for failures
Deployment Targets:
- Set of EC2 intances with tags
- Directly to an Auto Scaling Group
- Mix of ASG / Tags so you can build deployjment segments
- Customization in scripts with DEPLOYMENT_GROUP_NAME environment variables
CodeDeploy Demo
0. appspec.yml file, app store in S3
version: 0.0 os: linux files: - source: /index.html destination: /var/www/html/ hooks: BeforeInstall: - location: scripts/install_dependencies timeout: 300 runas: root - location: scripts/start_server timeout: 300 runas: root ApplicationStop: - location: scripts/stop_server timeout: 300 runas: root
1. Create two Roles
One for CodeDeploy role:
Another for EC2 role, because EC2 need to pull the code from S3
2. Create an EC2 instance where CodeDeploy can deploy to:
- Add IAM Role for EC2
- Add HTTP for security group
- Add Tag "Enviroment=Dev"
3. SSH into EC2 instance
sudo yum update sudo yum install ruby // download the agent wget https://aws-codedeploy-eu-west-3.s3.amazonaws.com/latest/install chmod +x ./install sudo ./install auto sudo service codedeploy-agent status // The AWS CodeDeploy agent is running as PID 7640
4. Create CodeDeploy Application
5. Create Deployment Group
6. Create Deployment
7. View the app in broswer by using EC2 Public IPV4 address.
RollBack
- You can specify automated roolback options
- Rollback when a deploymetn fails
- Disable rollbacks - Do not perform rollbacks fro this deployement
- If a roll back happens, CodeDepoy redepploys the last know good revision as a new deployment. (So it will get a new deployment ID).
CodeStar
'
Ref: https://aws.amazon.com/blogs/devops/how-to-enable-caching-for-aws-codebuild/
Without cache:
Enable cache:
BUILD time was faster. As the dependencies didn’t need to get downloaded, but were reused from cache.
Enable S3 and CloudWatch Logs integration - AWS CodeBuild monitors functions on your behalf and reports metrics through Amazon CloudWatch. These metrics include the number of total builds, failed builds, successful builds, and the duration of builds. You can monitor your builds at two levels: Project level, AWS account level. You can export log data from your log groups to an Amazon S3 bucket and use this data in custom processing and analysis, or to load onto other systems.
Incorrect options:
Use CloudWatch Events - You can integrate CloudWatch Events with CodeBuild. However, we are looking at storing and running queries on logs, so Cloudwatch logs with S3 integration makes sense for this context.
Deployment Groups: Goups to tagged intances (prod/dev/test)
You can specify one or more deployment groups for a CodeDeploy application. The deployment group contains settings and configurations used during the deployment. Most deployment group settings depend on the compute platform used by your application. Some settings, such as rollbacks, triggers, and alarms can be configured for deployment groups for any compute platform.
In an EC2/On-Premises deployment, a deployment group is a set of individual instances targeted for deployment. A deployment group contains individually tagged instances, Amazon EC2 instances in Amazon EC2 Auto Scaling groups, or both.
Amazon ECR users require permission to call ecr:GetAuthorizationToken before they can authenticate to a registry and push or pull any images from any Amazon ECR repository. Amazon ECR provides several managed policies to control user access at varying levels