Sourcepad Staging Server Setup
[SP setup] vs
VPC/Virtual Private Cloud - Each Sourcepad project has a staging VPC and a production VPC. This is a set of virtual machines (which Amazon calls instances) which make up the project.
VPC Components (Each of these are usually their own virtual machines)
|Component||What Does It Do||Comparison to Local Environment||Privacy|
|Bastion||This is the only machine that has shell access to the other machines||Iterm||Public|
|Web Server/Nginx/Load Balancer||Routes
|App Server||Runs the Rails app||
|AWS Relational Database Service (RDS)||Contains the relational database.||Your local RDBMS engine (PostgreSQL or MySQL)||Private|
|AWS ElastiCache (Optional)||Contains a key-value store (usually Redis). Redis is usually used for job queues like Resque and as a caching layer for your Rails app.||Your local Redis/
|AWS Route 53/DNS/Domain Naming System (Optional)||An alternative to other DNS providers like
|You can choose to not set this up, but setting this up allows you to have one admin panel for both managing instances and managing DNS stuff.|
|AWS Simple Storage Service (S3)||Stores basically anything (photos, reports/PDFs, file uploads)||Your hard drive.||Private|
|NAT/Network Address Translation||Grants Internet access to the rest of the VPC.||Your Wifi modem.||Public|
Why We Split Each Role Up
Theoretically, all of this can be in one machine, but we do not put it in one machine, because:
- Multiple points of failure–we want to split things up so that even if things like the caching layer goes down, the web app can still function because the web app and RDS machine is still up.
- Can scale specific parts of the app–different machines have different requirements. Bastion doesn’t need much, it literally is just a platform for deploying the apps, so we can leave it at a micro-level instance. A Rails server needs more CPU and an RDS server needs more memory (database accesses are memory intensive). If we need a faster Rails server, we do not need to scale the other components up.
- Continuing with different requirements for each machine, there are also different prices for each Amazon service.
- Even if you have your staging server’s database username and password, you won’t be able to access them from your local machine. Why is that? This is because your staging server’s VPC only allows very few hosts to have shell access.
- These hosts are the Bastion server(s) (usually just one for staging).
Accessing the Private Instances
This will not work:
# You are not inside the Bastion host yet. $ ssh [email protected] # ssh: connect to host 10.10.10.10 port 22: Operation timed out
When you are deploying your code via Capistrano, your Capistrano settings, if they have been set by another SP developer, already do this process (so you can deploy with ease). To confirm this, check out
/config/deploy/staging.rb or config/deploy/production.rb.
2 primary ways to scale:
- Vertical scaling: Increasing the computing power of a machine (either CPU or memory). Most probably, you’ll do this with RDS servers first.
- Horizontal scaling: Increasing the number of machines that do a specific role. Most probably, you’ll do this with app servers first.
Example of a project that is vertically scaled: WF. WF needs a big database since it has millions of database rows. It is vertically scaled in the RDS layer (it is also horizontally scaled hehe).
Example of a project that is horizontally scaled: ?.
This is why it is better to have efficient code since efficient code is cheaper, computing wise. You don’t need to scale immediately :).
- EC2 instances: Bastion, nginx, App, front-end. No hard drives yet.
- EBS Volumes:.
- Elastic IP: Static IPs of the EC2 instances. If you do not have this, if you restart server, new address.
- Elastic Beanstalk: Easiest to run and like Heroku.
- OpsWorks: You need chef recipes.
- Full management: CloudFormation.
- Identity and Access Management–per user.
Creating a VPC
- IPv4 - 10.0.0.0/16
- VPS/Virtual Private Server – old way.
- NAT is automatically included.
- Why not Digital Ocean/Heroku? Because they are also attached to AWS, and you are hiring the labor.
- RDS needs 2 subnets. Public - 0, first - 1, second - 2. First private subnet: web server/front-end. Second private subnet: RDS/Redis.
- Security Groups - pota hahaha.
Create Security Group - Network/Seceurity Group.
- Create Security Group - Source is the Private IPs.
- Type PostgreSQL.
- Type. 5432 PostgreSQL. 3306 MySQL. 6379 Redis.
- Default VPC security group firewall of ISP.
- Create Security Group for PostgreSQL, another SG for Bastion–Type SSH, protocol TCP, port range 22, source. Sometimes anywhere.
- Create SG for Nginx. TCP 80, TCP 443, TCP 22, anywhere.
- Create SG for API. Port 3000.
NAT: Source: Itself. From the cloud to itself.
- Private IP,
- Creating an RDS.
Bike workouts: Need to eat before hand, you will bonk like yesterday. Post-workout should really just be tofu, eggs, some...