Securing Docker Containers
Localhost Development is like a Private Parking Garage
It’s easy to become complacent when you do a lot of development on localhost. It’s great to have a machine where that lets you bypass hosting fees, deployment, etc. Development on localhost is like having a private garage. You can leave the car unlocked, hell you can leave the doors open, it’s a private garage, so no one is getting in. But, would you expose your vehicle like that in a public parking lot? You might, until it gets damaged or stolen. I recently learned my lesson and wanted to share it with all of you.
The Digital Ocean is Full of Predators
I use Digital Ocean to host my app. I’ll eventually switch over to AWS, but I like the simplicity of Digital Ocean, and I like their branding. I like the idea of having “droplets” in a “digital ocean.” I’ve been so used to localhost development that I’ve been ignorant to the best practices when Dockerizing an application and deploying it to a “public” domain, the “digital ocean.”
One day, I noticed strange behavior with my app. Sometimes, it’d work, and I’d be able to hit API endpoints, but sometimes it wouldn’t. I went through standard troubleshooting, but couldn’t find any reason why I couldn’t hit my API. Often, the most sinister attacks are hidden in plain sight. It wasn’t until I checked the logs in my PostgreSQL container that I found something…strange.
Kinsing Malware
Simply put, Kinsing Malware is a sneaky trojan horse of a virus that exploits insecure software to mine cryptocurrency. I basically broke every rule in the book which made my a prime target. I was using a Ubuntu-based Digital Ocean Droplet, insecure ports, and weak, vulnerable credentials.
I discovered the malware by inspecting the logs for my PostgreSQL container and saw that a process was trying to mount a volume to it, but luckily it failed. The scary part is the app mostly behaved, but it was acting strange enough to be noticeable, but not everyone might have caught it.
Based on my research, Kinsing is a malware written in GO that targets containers with misconfigured Docker API ports (guilty.) What it does is use the API vulnerability to instantiate Ubuntu to download a script (wget) and executes it periodically with via cron job. It uses a volumne mounted to /dev/null to store the shell script.
Lock it All Down
Passwords
First, and it goes without saying, use stronger passwords. With weak passwords, the Kinsing malware will be able to spread from the instantiated Ubuntu container via the Docker Daemon API and brute force it’s way to other containers, leeching their resources for its own bidding. Securing your passwords is low-hanging, but effective fruit.
Firewall
Here were my vulnerable firewall settings
2/tcp ALLOW Anywhere
2375/tcp ALLOW Anywhere
2376/tcp ALLOW Anywhere
80 ALLOW Anywhere
443 ALLOW Anywhere
Nginx Full ALLOW Anywhere
80/tcp ALLOW Anywhere
443/tcp ALLOW Anywhere
22/tcp (v6) ALLOW Anywhere (v6)
2375/tcp (v6) ALLOW Anywhere (v6)
2376/tcp (v6) ALLOW Anywhere (v6)
80 (v6) ALLOW Anywhere (v6)
443 (v6) ALLOW Anywhere (v6)
Nginx Full (v6) ALLOW Anywhere (v6)
80/tcp (v6) ALLOW Anywhere (v6)
443/tcp (v6) ALLOW Anywhere (v6)
The ports for the Docker API (2375/tcp and 2376/tcp) were completely exposed, making it publically accessible 🤦♂️
Also, my SSH port was open to the public, lol. 🤦♂️
Here’s how I fixed it
# Remove Docker API ports
sudo ufw delete allow 2375/tcp
sudo ufw delete allow 2376/tcp
# Remove redundant web server rules (keep only what you need)
sudo ufw status numbered
# Then delete redundant rules using their numbers
sudo ufw delete [rule_number]
# Allow SSH (preferably from specific IP addresses)
sudo ufw delete allow 22/tcp
sudo ufw allow from your_ip_address to any port 22 proto tcp
# Allow web traffic
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
# Deny direct access to internal services
sudo ufw deny 5432/tcp
sudo ufw deny 6379/tcp
sudo ufw deny 8080/tcp
sudo ufw deny 8081/tcp
sudo ufw deny 3000/tcp
# First, reset your UFW to start clean
sudo ufw reset
# Enable UFW
sudo ufw default deny incoming
sudo ufw default allow outgoing
# Allow SSH (ideally from specific IP addresses only)
sudo ufw allow from your_ip_address to any port 22 proto tcp
# Allow web traffic
sudo ufw allow 80,443/tcp comment 'Nginx web traffic'
# Deny direct access to internal services
sudo ufw deny 5432/tcp comment 'PostgreSQL'
sudo ufw deny 6379/tcp comment 'Redis'
sudo ufw deny 8080/tcp comment 'Adminer'
sudo ufw deny 8081/tcp comment 'Websocket'
sudo ufw deny 3000/tcp comment 'Backend'
# Enable the firewall
sudo ufw enable
# Verify your settings
sudo ufw status verbose
Fail2Ban
Another precaution I took was to install Fail2Ban which basically blacklists ip-addresses that have tried and failed to authenticate to services. This is good for attacks like Kinsing that may try to brute-force their way into your other containers.
# Install fail2ban if not already installed
sudo apt install fail2ban
# Configure and enable it
sudo systemctl enable fail2ban
sudo systemctl start fail2ban
Port Restrictions & Reverse Proxy
I also mistakenly had all the ports in my docker-compose publically facing. So, I locked those down by mapping the ports to localhost, or within the digital ocean’s droplet’s network, and not the entire internet.
ports:
- "127.0.0.1:5432:5432" # Restrict PostgreSQL to localhost
- "127.0.0.1:6379:6379" # Restrict Redis to localhost
- "127.0.0.1:8080:8080" # Restrict Adminer to localhost
Docker Compose Configuration
Docker Compose also provides a security option no-new-privileges:true. Basically, this prevents an escalation of priveleges, which is a great measure against Kinsing, as this will protect your containers from granting Kinsing elevated priveleges
Rest Easy(ier)
Don’t make the same mistakes that I did and rest easy!