Baldeep Hira's Blog

bay area programmer

 Home
About

Microservices on EC2

Network and deployment configuration for setting up microservices with http proxy / load balancer and bastion on EC2. This involves securing the EC2 instance and ensuring client requests are routed correctly to http proxy / load balancer. We need to setup a common VPC for all the ec2 instances and then create a public subnet that allows public inbound requests, enable Internet Gateway for that, and add routing table entries to ensure incoming requests are propogated. Next step is to setup a Security Group that allows inbound traffic for HTTP and HTTPS. Allocate elastic IP address for http proxy that provides static IP address. Then, we setup bastion server that is used to securely access all nodes internally using ssh.

Outline of steps:

  1. Create key pair
  2. Create VPC
  3. Create internet gateway
  4. Create public subnet
  5. Edit public-subnet routing table
  6. Create bastion security group
  7. Create webserver security group
  8. Create server-pool security group
  9. Launch EC2 instance
  10. Allocate elastic IP address
  11. Understanding elastic IP pricing
  12. Create user account and RSA private key
  13. Change hostname
  14. Install nginx
  15. Install mariadb
  16. Configure systemd service

Create key pair

  • Open Amazon aws console
  • Goto EC2 dashboard
  • In navigation pane, select Network & Security > Key Pairs
  • Choose Create key pair
  • For Name, enter descriptive name
  • For Key pair type, choose RSA
  • For Private key file format, choose pem
  • Click Create key pair button
  • Save the aws-key-pair-xyz.pem file on local computer in .ssh folder
  • chmod 400 aws-key-pair-xyz.pem

Create VPC

  • Open Amazon aws console
  • Goto VPC dashboard
  • In navigation pane, select virtual private cloud > Your VPCs
  • Click Create VPC button
  • Enter name tag as my-vpc
  • Enter IPv4 CIDR value as 172.31.0.0/16
  • For IPv6 CIDR, choose Amazon-provided IPv6 CIDR block
  • Click Create VPC button

Create internet gateway

  • Open Amazon aws console
  • Goto VPC dashboard
  • In navigation pane, select virtual private cloud > Internet gateways
  • Click Create internet gateway button
  • For Name tag, enter my-igw
  • Click Create internet gateway button
  • Click on newly created internet gateway, to edit
  • Click on Actions button, choose Attach to VPC
  • For Available VPCs, choose my-vpc
  • Click Attach internet gateway button

Create public subnet

  • Open Amazon aws console
  • Goto VPC dashboard
  • In navigation pane, select virtual private cloud > Subnets
  • Click Create subnet button
  • For VPC ID, choose my-vpc
  • For Subnet name, enter my-public-subnet
  • For Availability Zone, choose No preference
  • For IPv4 CIDR block, enter 172.31.93.0/24 (24 is more restrictive than 16)
  • For IPv6 CIDR, choose CIDR block from VPC
  • Click Create subnet button

Edit public-subnet routing table

  • Open Amazon aws console
  • Goto VPC dashboard
  • In navigation pane, select virtual private cloud > Routing tables
  • Select routing table associated with my-vpc
  • Under Routes tab, click on Edit routes button
  • Click Add route, enter 0.0.0.0/0 as Destination, enter my-igw as Target
  • Click Add route, enter ::/0 as Destination, enter my-igw as Target
  • Click Save changes button
  • Goto Subnet associations tab, click on Edit subnet associations button
  • Choose my-public-subnet from list, click Save associations button
  • Goto Tags tab, click on Manage tags button, click Add new tag button
  • For Key, enter Name. For Value, enter my-public-routes
  • Click Save button

Create bastion security group

  • Open Amazon aws console
  • Goto EC2 dashboard
  • In navigation pane, select Network & Security > Security Groups
  • Click Create security group button
  • For Security group name, enter bastion
  • For description, enter allow ssh access for developers
  • For VPC value, choose my-vpc
  • Goto Inbound rules section
  • Click Add rule, choose SSH for Type, Anywhere-IPv4 for Source
  • Click Create security group button

Create webserver security group

  • Open Amazon aws console
  • Goto EC2 dashboard
  • In navigation pane, select Network & Security > Security Groups
  • Click Create security group button
  • For Security group name, enter webserver
  • For description, enter allow http/https traffic
  • For VPC value, choose my-vpc
  • Goto Inbound rules section
  • Click Add rule, choose HTTP for Type, Anywhere-IPv4 for Source
  • Click Add rule, choose HTTP for Type, Anywhere-IPv6 for Source
  • Click Add rule, choose HTTPS for Type, Anywhere-IPv4 for Source
  • Click Add rule, choose HTTPS for Type, Anywhere-IPv6 for Source
  • Click Add rule, choose All traffic for Type, Custom - bastion for Source
  • Click Create security group button

Create server-pool security group

  • Open Amazon aws console
  • Goto EC2 dashboard
  • In navigation pane, select Network & Security > Security Groups
  • Click Create security group button
  • For Security group name, enter server-pool
  • For description, enter allow inbound http from webserver
  • For VPC value, choose my-vpc
  • Goto Inbound rules section
  • Click Add rule, choose Custom TCP for Type, 5000-6000 for Port range, Custom - webserver for Source
  • Click Add rule, choose All traffic for Type, Custom - bastion for Source
  • Click Add rule, choose All traffic for Type, Custom - server-pool for Source
  • Click Create security group button

Launch EC2 instance

  • Open Amazon aws console
  • Goto EC2 dashboard
  • In navigation pane, select Instances > Instances
  • Click Launch instances button
  • For Name, enter webServer
  • For Application and OS Images, select Amazon Linux
  • For Architecture, choose 64-bit (Arm)
  • For Instance type, choose t4g.small
  • For Key pair, select from existing value
  • For Network settings, click Edit button
  • For VPC, choose my-vpc
  • For Subnet, choose my-public-subnet
  • For Auto-assign public IP, choose Disable
  • For Firewall, choose Select existing security group
  • For Common security groups, choose server-pool
  • Under Advanced network configuration, for IPv6 IPs, set IPs to 1
  • For Configure storage, choose 8 GiB gp3
  • Click Launch instance button

Allocate elastic IP address

  • Open Amazon aws console
  • Goto EC2 dashboard
  • In navigation pane, select Network & Security > Elastic IPs
  • Click Allocate Elastic IP address button
  • For Public IPv4 address pool, choose Amazon’s pool of IPv4 addresses
  • Click Add new tag button, enter Name as Key, webserverIP as Value
  • Click Allocate button
  • Select newly created Elastic IP, webServerIP
  • Click Associate Elastic IP address button
  • For Resource type, choose Instance
  • For Instance, choose webServer
  • For Private IP address, choose value from dropdown
  • Click Associate button
  • Click on ec2 instance name
  • Click Actions > Networking > Manage IP addresses
  • Under IPv6 addresses, click Assign new IP address button
  • Leave IPv6 address field blank, click Save button, click Confirm button

Understanding elastic IP pricing

  • Elastic IP addresses are static and remain assigned to the instance regardless of instance state (running, stopped, terminated)
  • Elastic IP address are free as long as the instance they are associated to is running
  • If the associated EC2 instance is stopped or terminated then AWS starts charging for Elastic IP address
  • Disassociate Elastic IP address from EC2 instance to stop charges
  • Public IP addresses are dynamic and automatically released when an ec2 instance is stopped or terminated. The instance will get a new IP address upon restart.
  • SSH, database calls and other network protocol access is more efficient over private IP addresses

Create user account and RSA private key

  • Use public IP address of instance
  • Use default user name for AMI (ec2-user)
  • Use the private key (.pem) that you specified during instance launch
  • Make sure .pem, config and other files in .ssh have 400 file permissions
  • ssh command will ignore .pem/config files, if permissions are not 400
chmod 400 aws-key-pair-xyz.pem
ssh -i aws-key-pair-xyz.pem ec2-user@instance-public-ip-addr
sudo adduser newuser
sudo su newuser
mkdir ~/.ssh
chmod 700 .ssh
touch ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys
vi ~/.ssh/authorized_keys

# paste your public rsa key
'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQClKsfkNkuSevGj3e...'

Change hostname

sudo hostnamectl set-hostname webserver-172-31-93-111
sudo reboot

Install nginx

sudo dnf search nginx
sudo dnf install nginx
sudo systemctl status nginx
sudo systemctl enable nginx
sudo mv ~/my_nginx.conf /etc/nginx/nginx.conf
sudo systemctl start nginx

Install mariadb

sudo dnf search mariadb
sudo dnf install mariadb105-server
sudo systemctl enable mariadb
sudo systemctl start mariadb
sudo systemctl status mariadb
sudo mysql_secure_installation
n #for unix_socket authentication
Y #for change root password
mysql -u root -p
show engines;
show variables like '%innodb%';

Configure systemd service

sudo vi /etc/systemd/system/my-service.service
sudo systemctl daemon-reload
sudo systemctl enable my-service
sudo systemctl start my-service

Systemd file for my-service

# file: /etc/systemd/system/my-service.service
[Unit]
Description=service for my-service
After=network.target
After=mariadb.service

[Service]
User=ec2-user
Group=ec2-user
WorkingDirectory=/home/ec2-user/my-dir
ExecStart=/home/ec2-user/datamovr/my-service
Restart=on-failure
RestartSec=5s

[Install]
WantedBy=multi-user.target
Posted on Jun 18, 2023 at 12:46