Using overclocked speeds with an Intel i7-8086K at 5.2GHz in Ubuntu Server

This tutorial demonstrates how to assure that Ubuntu Server 18.04 x64 will operate at a specific CPU frequency overclock. I will also demonstrate how to validate the frequency per CPU core and monitor the temperature of each CPU core in near real time.

This tutorial presumes you have already installed and updated the OS, in addition to having a stable overclock. In my case, I have found that my Intel i7-8086K operates at 5.2GHz at 1.392 volts while my RAM is using its XMP profile. This tutorial, however, only focuses on CPU frequency settings. Hyper-threading is disabled in BIOS so Ubuntu sees 6 cores and 6 threads.

sudo apt install cpufrequtils lm-sensors -y

sudo vim /etc/init.d/cpufrequtils

Edit and enable these lines to your target overclocked frequency:

ENABLE="true"
GOVERNOR="performance"
MAX_SPEED="5200000"
MIN_SPEED="5200000"

5.0GHz = 5000000
5.2GHz = 5200000
5.4GHz = 5400000

Note: If AVX is set to 2 in BIOS, even with this MIN_SPEED setting, Ubuntu will cap itself at 5.0GHz. Set AVX to 0 to assure Ubuntu will stay at 5.2GHz constantly.

sudo service cpufrequtils restart

Open a new terminal and watch the CPU frequencies for cores 0-5 (6 cores)

watch -n.1 "cat /proc/cpuinfo | grep \"^[c]pu MHz\""

Find the available temp sensors on your system and say YES to everything:

sudo sensors-detect

Open a new terminal and watch the CPU temps:

watch -n.1 "sensors"

We can use OpenSSL to burn the CPU for testing… this “-multi 6” flag will burn all 6 of my cores (be sure you have proper cooling on your CPU!):

openssl speed -multi 6

Note: Like in Windows, if my CPU is not properly cooled resulting in CPU temps at or above 100 degree C, my multiplier will automatically throttle down in order to keep the core temps down below critical. If you’re needing to test single-threaded performance, “-multi 1” will just burn one core. Burning half the available cores or less will keep temps down quite a bit since heat dissipation on the IHS is cut in half, meaning the multiplier should stay at the set x52. This is all dependent on proper cooling.

Also checkout CoreFreq for seeing greater CPU details like your core multipliers:

https://github.com/cyring/CoreFreq

sudo apt install dkms git libpthread-stubs0-dev -y

git clone https://github.com/cyring/CoreFreq.git

cd CoreFreq

make

Load the kernel module:

sudo insmod corefreqk.ko

Load the daemon

sudo ./corefreqd

In a new window:

./CoreFreq/corefreq-cli

Configurations that didn’t help me, especially after reboots:

sudo bash -c 'for i in {0..5}; do cpufreq-set -c $i -g performance; done'

for x in /sys/devices/system/cpu/*/cpufreq/; do echo 5200000 | sudo tee $x/scaling_max_freq; done

for x in /sys/devices/system/cpu/*/cpufreq/; do echo 5200000 | sudo tee $x/scaling_min_freq; done

sudo cpupower frequency-set -d 5.2GHz -u 5.2GHz -g performance -r

sudo cpufreq-set -d 5.2GHz -u 5.2GHz -g performance -r

building caddy from source (and updating caddy on xenial)

sudo add-apt-repository ppa:gophers/archive
sudo vim /etc/apt/sources.list.d/gophers-ubuntu-archive-xenial.list

deb tor+http://ppa.launchpad.net/gophers/archive/ubuntu xenial main

sudo apt update && sudo apt install golang-1.10*
sudo apt remove golang-1.6*
sudo apt update && sudo apt dist-upgrade -V && sudo apt autoremove -y && sudo apt autoclean
/usr/lib/go-1.10/bin/go version
echo 'PATH="/usr/lib/go-1.10/bin:$PATH"' >> ~/.profile
source ~/.profile
go version

go version go1.10 linux/amd64

go get -u github.com/mholt/caddy
go get -u github.com/caddyserver/builds
cd ~/go/src/github.com/mholt/caddy/caddy
go run build.go -goos=linux -goarch=amd64
sudo service caddy stop
sudo cp /usr/local/bin/caddy /usr/local/bin/caddy.bak
cp ~/go/src/github.com/mholt/caddy/caddy/caddy /usr/local/bin
sudo setcap 'cap_net_bind_service=+ep' /usr/local/bin/caddy
caddy --version

Caddy 0.11.0 (+60a0208) (unofficial)

sudo shutdown -r now

A+ TLS config for ubuntu + nginx

These are my config notes for getting a brand new Xenial + nginx server online.

Install Tor:

sudo apt install tor apt-transport-tor
sudo gpg --keyserver keys.gnupg.net --recv 886DDD89

sudo gpg --export A3C4F0F979CAA22CDBA8F512EE8CBC9E886DDD89 | sudo apt-key add -

Edit the sources list by removing all the lines and adding these:

sudo vim /etc/apt/sources.list
deb tor+https://deb.torproject.org/torproject.org xenial main
deb tor+https://mirrors.wikimedia.org/ubuntu/ xenial main restricted universe multiverse
deb tor+https://mirrors.wikimedia.org/ubuntu/ xenial-updates main restricted universe multiverse
deb tor+https://mirrors.wikimedia.org/ubuntu/ xenial-security main restricted universe multiverse

Update the repos:

sudo add-apt-repository ppa:nginx/development
sudo add-apt-repository ppa:ondrej/nginx
sudo add-apt-repository ppa:ondrej/php
sudo add-apt-repository ppa:certbot/certbot

Add “tor+” to all of the above sources files in /etc/apt/sources.list.d/*

Update and restart:

sudo apt update && sudo apt upgrade -V && sudo apt autoremove -y && sudo shutdown -r now

Install nginx + certbot:

sudo apt install python-certbot-nginx -V

Add server_name to (replacing “_”):

sudo vim /etc/nginx/sites-available/default
server_name domain.net;

Get Let’s Encrypt cert for nginx:

sudo certbot --nginx -d domain.net --redirect --rsa-key-size 4096

Further harden the TLS config:

sudo vim /etc/letsencrypt/options-ssl-nginx.conf
ssl_ecdh_curve secp384r1;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;
ssl_session_tickets off;
ssl_stapling on;
ssl_stapling_verify on;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_prefer_server_ciphers on;
ssl_ciphers "ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:!3DES:!aNULL:!DES:!DSS:!eNULL:!EXP:!IDEA:!LOW:!MD5:!PSK:!RC4:!SEED";

Delete the “SSL” config:

sudo vim /etc/nginx/nginx.conf

Edit the nginx config:

sudo vim /etc/nginx/sites-available/default

replace “domain.net”

server {
        listen 80 default_server;
        listen [::]:80 default_server;

        server_name domain.net www.domain.net;
        return 301 https://$host$request_uri;

        server_tokens off;
        add_header X-Frame-Options SAMEORIGIN;
        add_header X-Content-Type-Options nosniff;
        add_header X-XSS-Protection "1; mode=block";
        add_header Referrer-Policy "no-referrer";
}

server {
        listen 443 ssl http2;
        listen [::]:443 ssl http2;

        server_name domain.net www.domain.net;
        root /var/www;
        index index.php index.html index.htm;

        ssl_certificate /etc/letsencrypt/live/domain.net/fullchain.pem;
        ssl_certificate_key /etc/letsencrypt/live/domain.net/privkey.pem;
        include /etc/letsencrypt/options-ssl-nginx.conf;
        ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;

        server_tokens off;
        add_header Strict-Transport-Security 'max-age=31536000; includeSubDomains; preload';
        add_header X-Frame-Options SAMEORIGIN;
        add_header X-Content-Type-Options nosniff;
        add_header X-XSS-Protection "1; mode=block";
        add_header Referrer-Policy "no-referrer";

        resolver 8.8.8.8 8.8.4.4 valid=300s;

# For WordPress

        location / {
        try_files $uri $uri/ /index.php?$args;
        }

        location ~ \.php$ {
                include snippets/fastcgi-php.conf;
                fastcgi_pass unix:/var/run/php/php7.2-fpm.sock;
                fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
                include fastcgi_params;
        }
}

Validate the nginx config:

sudo nginx -t

Restart nginx:

sudo service nginx restart

Add inbound and outbound firewall rules:

sudo ufw limit 22/tcp && sudo ufw allow 443/tcp && sudo ufw allow out 22/tcp && sudo ufw allow out 25/tcp && sudo ufw allow out 53/udp && sudo ufw allow out 443/tcp && sudo ufw allow out 9050/tcp && sudo ufw deny out to any && sudo ufw enable && sudo ufw status verbose

Updating Caddy to PHP 7.2

This was wonderfully easy.

sudo apt update && sudo apt dist-upgrade -y && sudo apt autoremove -y && sudo apt autoclean
sudo apt install php7.2-cli php7.2-common php7.2-fpm php7.2-json php7.2-mysql php7.2-opcache php7.2-readline

Update Caddyfile:

sudo vim /etc/caddy/Caddyfile
www.yawnbox.com {
        redir https://yawnbox.com{uri}
        }

yawnbox.com {
        root /var/www/yawnbox/
        log stdout
        errors stderr

header / {
        Content-Security-Policy "default-src 'self'; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline';"
        Expect-CT "enforce; max-age=7073887;"
        Referrer-Policy "strict-origin, strict-origin-when-cross-origin"
        Strict-Transport-Security "max-age=15768000; includeSubDomains; preload"
        X-XSS-Protection "1; mode=block"
        X-Content-Type-Options "nosniff"
        X-Frame-Options "DENY"
        }

fastcgi / /var/run/php/php7.2-fpm.sock {
        ext .php
        split .php
        index index.php
        }

rewrite / {
        to {path} {path}/ /index.php?{query}
        }

tls {
        protocols tls1.2
        curves p384
        key_type p384
        }
sudo service caddy restart
sudo apt remove php-mysql php7.1-cli php7.1-common php7.1-fpm php7.1-json php7.1-mysql php7.1-opcache php7.1-readline

Ubuntu SSH crypto hardening

Fix the security and privacy of your Ubuntu 16.04, 16.10, and 17.04 web server access. Fuck global mass surveillance.

Special thanks to @stribika for writing a very similar guide two years ago.

From “man sshd_config”

allowable ciphers

Run “ssh -Q cipher” for validating usable “Ciphers” on clients and servers

Specifies the ciphers allowed.  Multiple ciphers must be comma-separated.  If the specified value begins with a ‘+’ character, then the specified ciphers will be appended to the default set instead of replacing them.

allowable message authentication code algorithms

Run “ssh -Q mac” for validating usable “MACs” on clients and servers

Specifies the available MAC (message authentication code) algorithms.  The MAC algorithm is used for data integrity protection. Multiple algorithms must be comma-separated.  If the specified value begins with a ‘+’ character, then the specified algorithms will be appended to the default set instead of replacing them. The algorithms that contain "-etm" calculate the MAC after encryption (encrypt-then-mac).  These are considered safer and their use recommended.

allowable key exchange algorithms

Run “ssh -Q kex” for validating usable “KexAlgorithms” on clients and servers

Specifies the available KEX (Key Exchange) algorithms.  Multiple algorithms must be comma-separated.  Alternately if the specified value begins with a ‘+’ character, then the specified methods will be appended to the default set instead of replacing them.

allowable server key algorithms

Run “ssh -Q key” for validating usable “HostKeyAlgorithms” on servers

Specifies the host key algorithms that the server offers.

allowable key authentication types

Run “ssh -Q key” for validating usable “HostbasedAcceptedKeyTypes” on servers

Specifies the key types that will be accepted for hostbased authentication as a comma-separated pattern list.  Alternately if the specified value begins with a ‘+’ character, then the specified key types will be appended to the default set instead of replacing them.

allowable public key authentication types

Run “ssh -Q key” for validating usable “PubkeyAcceptedKeyTypes” on servers

Specifies the key types that will be accepted for public key authentication as a comma-separated pattern list.  Alternately if the specified value begins with a ‘+’ character, then the specified key types will be appended to the default set instead of replacing them.

Fix your server keys

cd /etc/ssh

sudo rm ssh_host_*key*

sudo ssh-keygen -t ed25519 -f ssh_host_ed25519_key -N "" < /dev/null

Amending sshd

sudo vim /etc/ssh/sshd_config

Only use the ed25519 key (delete the others):

HostKey /etc/ssh/ssh_host_ed25519_key

Add these lines (tailor them down based on what you know your client and server have available (see above for "ssh -Q x" options)):

Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr

MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512,hmac-sha2-256

KexAlgorithms curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521

HostKeyAlgorithms ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519

HostbasedAcceptedKeyTypes ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519

PubkeyAcceptedKeyTypes ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519

Restart sshd:

sudo service ssh restart

ssh over Tor

Fix metadata leaks by using Tor as your second end-to-end encrypted tunnel if you don't mind a mildly delayed CLI due to added latency.

Install Tor by first fixing apt sources and adding Tor's repo:

sudo vim /etc/apt/sources.list

Delete all lines and use these (replace "zesty" if needed):

deb https://mirrors.wikimedia.org/ubuntu/ zesty main restricted universe multiverse
deb https://mirrors.wikimedia.org/ubuntu/ zesty-updates main restricted universe multiverse
deb https://mirrors.wikimedia.org/ubuntu/ zesty-backports main restricted universe multiverse
deb https://mirrors.wikimedia.org/ubuntu/ zesty-security main restricted universe multiverse
deb https://deb.torproject.org/torproject.org zesty main

Install Tor's signing key:

sudo gpg --keyserver keys.gnupg.net --recv A3C4F0F979CAA22CDBA8F512EE8CBC9E886DDD89

sudo gpg --export A3C4F0F979CAA22CDBA8F512EE8CBC9E886DDD89 | sudo apt-key add -

Update and install Tor:

sudo apt-get update && sudo apt-get install tor deb.torproject.org-keyring -y

Configure Tor for an onion:

sudo vim /etc/tor/torrc

Delete all lines and add these:

HiddenServiceDir /var/lib/tor/hidden_service/
HiddenServicePort 22 127.0.0.1:22

Restart Tor:

sudo service tor restart

Find your new dot-onion address:

sudo cat /var/lib/tor/hidden_service/hostname

Configure sshd to only listen via Tor (and not exposed on the clear net):

sudo vim /etc/ssh/sshd_config

Add (or change) this line:

ListenAddress 127.0.0.1:22

Restart sshd:

sudo service ssh restart

Firewall everything

Presuming you are only hosting a web server over ports 80 and 443:

sudo ufw allow 80/tcp && sudo ufw allow 443/tcp && sudo ufw allow out 53/udp && sudo ufw allow out 80/tcp && sudo ufw allow out 123/udp && sudo ufw allow out 443/tcp && sudo ufw allow out 9050/tcp && sudo ufw deny out to any && sudo ufw enable && sudo ufw status verbose

80 for http
443 for https
53 out for DNS
123 out for NTP
9050 out for Tor

Deny everything else.

client side for Tor

sudo vim /etc/ssh/ssh_config

Add these lines under "Host *" (tailor the Ciphers, MACs, and Kex down based on what you know your client and server have available (see above for "ssh -Q x" options)):

UseRoaming no

proxyCommand ncat -v --proxy localhost:9050 --proxy-type socks5 %h %p

Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr

MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512,hmac-sha2-256

KexAlgorithms curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521

Generate client keys:

ssh-keygen -t ed25519 -o -a 100

Restart ssh:

sudo service ssh restart

Send the client public key to the server:

ssh-copy-id yawnbox@2vytis5xf5djnaoo.onion

Connect to the server with debug to verify hardened crypto:

ssh -v yawnbox@2vytis5xf5djnaoo.onion

You will find this info buried:

debug1: kex: algorithm: curve25519-sha256@libssh.org
debug1: kex: host key algorithm: ssh-ed25519
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC:  compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC:  compression: none

debug2: key: /home/yawnbox/.ssh/id_rsa ((nil))
debug2: key: /home/yawnbox/.ssh/id_dsa ((nil))
debug2: key: /home/yawnbox/.ssh/id_ecdsa ((nil))
debug2: key: /home/yawnbox/.ssh/id_ed25519 (0x55zg8nba8f20)

Cheers

bonus server config script


#!/bin/bash

sudo apt-get update

sudo apt-get upgrade -y

sudo apt-get dist-upgrade -y

sudo apt-get install tor openssh-server -y

sudo apt-get autoremove -y

sudo apt-get autoclean

cd /etc/ssh

sudo rm ssh_host_*key*

sudo ssh-keygen -t ed25519 -f ssh_host_ed25519_key -N "" < /dev/null

cd ~

sudo mv /etc/ssh/sshd_config /etc/ssh/sshd_config.bak

sudo touch /etc/ssh/sshd_config

sudo echo "Port 22" >> /etc/ssh/sshd_config

sudo echo "HostKey /etc/ssh/ssh_host_ed25519_key" >> /etc/ssh/sshd_config

sudo echo "SyslogFacility AUTH" >> /etc/ssh/sshd_config

sudo echo "LogLevel INFO" >> /etc/ssh/sshd_config

sudo echo "LoginGraceTime 30" >> /etc/ssh/sshd_config

sudo echo "PermitRootLogin no" >> /etc/ssh/sshd_config

sudo echo "StrictModes yes" >> /etc/ssh/sshd_config

sudo echo "MaxAuthTries 5" >> /etc/ssh/sshd_config

sudo echo "MaxSessions 5" >> /etc/ssh/sshd_config

sudo echo "PasswordAuthentication no" >> /etc/ssh/sshd_config

sudo echo "PermitEmptyPasswords no" >> /etc/ssh/sshd_config

sudo echo "ChallengeResponseAuthentication no" >> /etc/ssh/sshd_config

sudo echo "UsePAM yes" >> /etc/ssh/sshd_config

sudo echo "X11Forwarding no" >> /etc/ssh/sshd_config

sudo echo "PrintMotd no" >> /etc/ssh/sshd_config

sudo echo "AcceptEnv LANG LC_*" >> /etc/ssh/sshd_config

sudo echo "Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr" >> /etc/ssh/sshd_config

sudo echo "MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512,hmac-sha2-256" >> /home/cs/test_sshd

sudo echo "KexAlgorithms curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521" >> /etc/ssh/sshd_config

sudo echo "HostKeyAlgorithms ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519" >> /etc/ssh/sshd_config

sudo echo "HostbasedAcceptedKeyTypes ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519" >> /etc/ssh/sshd_config

sudo echo "PubkeyAcceptedKeyTypes ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519" >> /etc/ssh/sshd_config

sudo service ssh restart

sudo mv /etc/apt/sources.list /etc/apt/sources1.bak

sudo touch /etc/apt/sources.list

sudo echo "deb https://mirrors.wikimedia.org/ubuntu/ xenial main restricted universe multiverse" >> /etc/apt/sources.list

sudo echo "deb https://mirrors.wikimedia.org/ubuntu/ xenial-updates main restricted universe multiverse" >> /etc/apt/sources.list

sudo echo "deb https://mirrors.wikimedia.org/ubuntu/ xenial-backports main restricted universe multiverse" >> /etc/apt/sources.list

sudo echo "deb https://mirrors.wikimedia.org/ubuntu/ xenial-security main restricted universe multiverse" >> /etc/apt/sources.list

sudo echo "deb https://deb.torproject.org/torproject.org xenial main" >> /etc/apt/sources.list

sudo torify gpg --keyserver keys.gnupg.net --recv A3C4F0F979CAA22CDBA8F512EE8CBC9E886DDD89

sudo torify gpg --export A3C4F0F979CAA22CDBA8F512EE8CBC9E886DDD89 | sudo apt-key add -

sudo apt-get update

sudo apt-get install tor deb.torproject.org-keyring apt-transport-tor -y

sudo mv /etc/apt/sources.list /etc/apt/sources2.bak

sudo touch /etc/apt/sources.list

sudo echo "deb tor+https://mirrors.wikimedia.org/ubuntu/ xenial main restricted universe multiverse" >> /etc/apt/sources.list

sudo echo "deb tor+https://mirrors.wikimedia.org/ubuntu/ xenial-updates main restricted universe multiverse" >> /etc/apt/sources.list

sudo echo "deb tor+https://mirrors.wikimedia.org/ubuntu/ xenial-backports main restricted universe multiverse" >> /etc/apt/sources.list

sudo echo "deb tor+https://mirrors.wikimedia.org/ubuntu/ xenial-security main restricted universe multiverse" >> /etc/apt/sources.list

sudo echo "deb tor+https://deb.torproject.org/torproject.org xenial main" >> /etc/apt/sources.list

sudo echo "capability dac_read_search," >> /etc/apparmor.d/abstractions/tor

sudo /etc/init.d/apparmor reload

sudo mv /etc/tor/torrc /etc/tor/torrc.bak

sudo touch /etc/tor/torrc

sudo echo "HiddenServiceDir /var/lib/tor/hidden_service/" >> /etc/tor/torrc

sudo echo "HiddenServicePort 22 127.0.0.1:22" >> /etc/tor/torrc

sudo service tor restart

sudo touch ~/onion.txt

sudo cat /var/lib/tor/hidden_service/hostname >> ~/onion.txt

sudo echo "ListenAddress 127.0.0.1:22" >> /etc/ssh/sshd_config

sudo service ssh restart

sudo ufw allow 22/tcp

sudo ufw allow 80/tcp

sudo ufw allow 443/tcp

sudo ufw allow out 22/tcp

sudo ufw allow out 53/udp

sudo ufw allow out 80/tcp

sudo ufw allow out 123/udp

sudo ufw allow out 443/tcp

sudo ufw allow out 9050/tcp

sudo ufw deny out to any

sudo ufw enable

sudo ufw status verbose

sudo cat ~/onion.txt

Tor onion service config fails due to apparmor

Thanks for the help, Will.

After installing Tor on a new host and configuring an onion service, Tor fails due to AppArmor.

Hosts:

Xenial server
Zesty server

Tor versions:

0.3.0.9
0.3.1.4-alpha

Errors:

/var/log/kern.log |grep tor

Jul 20 19:25:58 zesty kernel: [   50.173406] audit: type=1400 audit(1500578758.127:16): apparmor="DENIED" operation="capable" profile="system_tor" pid=2148 comm="tor" capability=2  capname="dac_read_search"

/var/log/syslog |grep tor

Jul 20 19:26:00 zesty tor[2190]: Jul 20 19:26:00.111 [notice] Tor 0.3.1.4-alpha (git-c3fe257c709bb814) running on Linux with Libevent 2.0.21-stable, OpenSSL 1.0.2g, Zlib 1.2.11, Liblzma N/A, and Libzstd N/A.
Jul 20 19:26:00 zesty tor[2190]: Jul 20 19:26:00.112 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning
Jul 20 19:26:00 zesty tor[2190]: Jul 20 19:26:00.113 [notice] This version is not a stable Tor release. Expect more bugs than usual.
Jul 20 19:26:00 zesty tor[2190]: Jul 20 19:26:00.114 [notice] Read configuration file "/usr/share/tor/tor-service-defaults-torrc".
Jul 20 19:26:00 zesty tor[2190]: Jul 20 19:26:00.114 [notice] Read configuration file "/etc/tor/torrc".
Jul 20 19:26:00 zesty tor[2190]: Configuration was valid
Jul 20 19:26:00 zesty tor[2193]: Jul 20 19:26:00.223 [notice] Tor 0.3.1.4-alpha (git-c3fe257c709bb814) running on Linux with Libevent 2.0.21-stable, OpenSSL 1.0.2g, Zlib 1.2.11, Liblzma N/A, and Libzstd N/A.
Jul 20 19:26:00 zesty tor[2193]: Jul 20 19:26:00.224 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning
Jul 20 19:26:00 zesty tor[2193]: Jul 20 19:26:00.225 [notice] This version is not a stable Tor release. Expect more bugs than usual.
Jul 20 19:26:00 zesty tor[2193]: Jul 20 19:26:00.225 [notice] Read configuration file "/usr/share/tor/tor-service-defaults-torrc".
Jul 20 19:26:00 zesty tor[2193]: Jul 20 19:26:00.226 [notice] Read configuration file "/etc/tor/torrc".
Jul 20 19:26:00 zesty tor[2193]: Jul 20 19:26:00.233 [warn] Directory /var/lib/tor/hidden_service/ cannot be read: Permission denied
Jul 20 19:26:00 zesty tor[2193]: Jul 20 19:26:00.234 [warn] Failed to parse/validate config: Failed to configure rendezvous options. See logs for details.
Jul 20 19:26:00 zesty tor[2193]: Jul 20 19:26:00.235 [err] Reading config failed--see warnings above.
Jul 20 19:26:00 zesty systemd[1]: tor@default.service: Main process exited, code=exited, status=1/FAILURE
Jul 20 19:26:00 zesty systemd[1]: tor@default.service: Unit entered failed state.
Jul 20 19:26:00 zesty systemd[1]: tor@default.service: Failed with result 'exit-code'.
Jul 20 19:26:00 zesty systemd[1]: tor@default.service: Service hold-off time over, scheduling restart.
Jul 20 19:26:00 zesty systemd[1]: tor@default.service: Start request repeated too quickly.
Jul 20 19:26:00 zesty systemd[1]: tor@default.service: Unit entered failed state.
Jul 20 19:26:00 zesty systemd[1]: tor@default.service: Failed with result 'exit-code'.

Solution

sudo vim /etc/apparmor.d/abstractions/tor

add this line to capabilities:

capability dac_read_search,

reload:

sudo /etc/init.d/apparmor reload
sudo service tor restart

Ubuntu Server + Caddy + Mediawiki

Ubuntu Server 16.04 + Caddy 0.10.2 + Mediawiki 1.28.2

For use on your private LAN (no LetsEncrypt).

sudo ufw limit 22/tcp && sudo ufw allow 80/tcp && sudo ufw allow out 22/tcp && sudo ufw allow out 25/tcp && sudo ufw allow out 53/udp && sudo ufw allow out 80/tcp && sudo ufw deny out to any && sudo ufw enable && sudo ufw status verbose
sudo vim /etc/apt/sources.list

Delete all lines, use these instead:

deb https://mirrors.wikimedia.org/ubuntu/ xenial main restricted universe multiverse
deb https://mirrors.wikimedia.org/ubuntu/ xenial-updates main restricted universe multiverse
deb https://mirrors.wikimedia.org/ubuntu/ xenial-backports main restricted universe multiverse
deb https://mirrors.wikimedia.org/ubuntu/ xenial-security main restricted universe multiverse
sudo apt-get update && sudo apt-get upgrade -y && sudo apt-get dist-upgrade -y && sudo shutdown -r now
sudo apt-get install mysql-server
mysql -u root -p
CREATE DATABASE mediawiki_db DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci;

At the least, change the user and password:

GRANT ALL ON mediawiki_db.* TO 'mediawiki_user'@'localhost' IDENTIFIED BY 'mediawiki_pass';
FLUSH PRIVILEGES;
EXIT;
sudo add-apt-repository ppa:ondrej/php
sudo apt-get update && sudo apt-get install php7.1-fpm php7.1-cli php-mysql php7.1-intl php7.1-curl php7.1-gd php7.1-mbstring php7.1-xml
curl https://getcaddy.com | bash && sudo chown root:root /usr/local/bin/caddy && sudo chmod 755 /usr/local/bin/caddy && sudo setcap 'cap_net_bind_service=+ep' /usr/local/bin/caddy && sudo mkdir /etc/caddy && sudo chown -R root:www-data /etc/caddy && sudo mkdir /etc/ssl/caddy && sudo chown -R www-data:root /etc/ssl/caddy && sudo chmod 0770 /etc/ssl/caddy
sudo vim /etc/caddy/Caddyfile
*:80 {
        root /var/www/
        log stdout
        errors stderr

header / {
        Content-Security-Policy "default-src 'self'; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline'"
        Referrer-Policy "strict-origin, strict-origin-when-cross-origin"
        X-XSS-Protection "1; mode=block"
        X-Content-Type-Options "nosniff"
        X-Frame-Options "DENY"
        }

fastcgi / /var/run/php/php7.1-fpm.sock {
        ext .php
        split .php
        index index.php
        }

rewrite / {
        to {path} {path}/ /index.php?{query}
        }
}
sudo chown www-data:www-data /etc/caddy/Caddyfile && sudo chmod 444 /etc/caddy/Caddyfile && sudo mkdir /var/www && sudo chown -R www-data:www-data /var/www && sudo chmod -R 555 /var/www && cd /tmp && wget https://releases.wikimedia.org/mediawiki/1.28/mediawiki-1.28.2.tar.gz && tar -xvzf mediawiki-1.28.2.tar.gz && sudo mv mediawiki-1.28.2/* /var/www/ && sudo chown www-data:www-data -R /var/www/
cd ~ && wget https://raw.githubusercontent.com/mholt/caddy/master/dist/init/linux-systemd/caddy.service && sudo cp caddy.service /etc/systemd/system/ && sudo chown root:root /etc/systemd/system/caddy.service && sudo chmod 644 /etc/systemd/system/caddy.service && sudo systemctl daemon-reload && sudo systemctl enable caddy.service && sudo /usr/local/bin/caddy -log stdout -agree=true -conf=/etc/caddy/Caddyfile -root=/var/tmp

Setup Mediawiki by navigating to the internal IP address of the server. Upon completion, download LocalSettings.php then paste its contents to:

sudo vim /var/www/LocalSettings.php

Restart the server:

sudo shutdown -r now

Caddy will automatically start and you can start using your wiki!

Moved from Apache to Caddy and RSA to EC TLS for WordPress

^ Qualys SSL Labs test for yawnbox.com

^ Security Headers (dot-IO) test for yawnbox.com

With very special thanks to this guide, Running WordPress with Caddy. I was also able to remove several unnecessary PHP applications that Apache needed.

Here’s my Caddyfile:

www.yawnbox.com {
        redir https://yawnbox.com{uri}
        }

yawnbox.com {
        root /var/www/
        log stdout
        errors stderr

header / {
	Content-Security-Policy "default-src 'self'; script-src 'self'; style-src 'self' 'unsafe-inline'"
	Referrer-Policy "strict-origin, strict-origin-when-cross-origin"
        Strict-Transport-Security "max-age=15768000; includeSubDomains; preload"
        X-XSS-Protection "1; mode=block"
        X-Content-Type-Options "nosniff"
        X-Frame-Options "DENY"
        }

fastcgi / /var/run/php/php7.1-fpm.sock {
        ext .php
        split .php
        index index.php
        }

rewrite / {
        to {path} {path}/ /index.php?{query}
        }

tls / {
        protocols tls1.2
        curves p384
        key_type p384
        }
}

Ubuntu OS updates with security and privacy

Never Forget DSA-3733
Validating signatures > MitM > RCE

The Debian developer community refused to implement transport crypto for updates because “signing packages is secure enough”. Utter bullshit.

This is a quick guide on how to dramatically improve the privacy and security of your Ubuntu web server. It requires the installation of “apt-transport-tor”, an application that will allow APT transfers to occur over Tor. There is also an application called “apt-transport-https” that is already installed in Ubuntu that we’ll use.

After reviewing the existing Ubuntu updates mirrors in the USA, I found that Wikimedia has a great TLS configuration. Please contribute to the Google Doc list!

First add Tor Project’s Debian/Ubuntu repository to your system for up-to-date Tor software: https://www.torproject.org/docs/debian.html.en

Then perform the following:

sudo apt-get update

sudo apt-get install apt-transport-tor

sudo vim /etc/apt/sources.list

Edit “sources.list” to just use only “deb”. “deb-src” is only needed if you build from source which most people do not. You can safely delete the deb-src lines from the file. Replace all of the default Ubuntu repos with Wikimedia’s and be sure to add “tor+” before the “https”. Doing so adds end-to-end encryption via HTTPS, and it becomes Torified meaning network adversaries will have a more difficult time analyzing what software and what versions of said software are installed on your web server.

deb tor+https://mirrors.wikimedia.org/ubuntu/ xenial main restricted universe multiverse
deb tor+https://mirrors.wikimedia.org/ubuntu/ xenial-updates main restricted universe multiverse
deb tor+https://mirrors.wikimedia.org/ubuntu/ xenial-backports main restricted universe multiverse
deb tor+https://mirrors.wikimedia.org/ubuntu/ xenial-security main restricted universe multiverse
deb tor+https://deb.torproject.org/torproject.org xenial main

All your future apt-get update, upgrade, and dist-upgrade commands will now be performed over Tor and using high-grade HTTPS.

Firewall changes

If you use UFW to manage your iptables firewall rules, and if you’re properly restricting outbound connections, below is what your config might change to. First reset your UFW rules:

sudo ufw reset

Then:

sudo ufw limit 22/tcp
sudo ufw allow 443/tcp
sudo ufw allow out 22/tcp
sudo ufw allow out 25/tcp
sudo ufw allow out 53/udp
sudo ufw allow out 443/tcp
sudo ufw allow out 9050/tcp
sudo ufw deny out to any
sudo ufw enable
sudo ufw status verbose

Or with one command:

sudo ufw limit 22/tcp && sudo ufw allow 443/tcp && sudo ufw allow out 22/tcp && sudo ufw allow out 25/tcp && sudo ufw allow out 53/udp && sudo ufw allow out 443/tcp && sudo ufw allow out 9050/tcp && sudo ufw deny out to any && sudo ufw enable && sudo ufw status verbose

This UFW (iptables) rule set makes it so brute forcing SSH won’t work and allows all incoming HTTPS traffic. These rules also make it so no traffic can leave the web server unless it is SSH (22), SMTP (25), DNS (53), HTTPS (443), or Tor Socks (9050) traffic. Most web servers do not go as far as block all outbound traffic by default, but it is important in case the web server does become compromised. I would usually allow outbound HTTP (80) traffic because the default Ubuntu update repositories require HTTP. However, we will be Torifying Apt so that’s why we allow outbound 9050/tcp. If you don’t want to Torify Apt, you’ll need to allow outbound 80/tcp instead of 9050/tcp.

Debian update repos: transport security and privacy

Some friends and I have started thinking about ways to bring attention to this important issue.

In short, network adversaries can view not only what repositories your server operating system connects to (metadata), when (metadata), but precisely which updates are being applied. This allows a network adversary to not only know what vulnerabilities your operating system and applications are vulnerable to, but they know what applications are installed and likely running.

This critical issue is aside from the fact that most repository signing keys are using 1024-bit keys, some of which created in 2004 and do not expire. This is also aside from the fact that there are many man-in-the-middle attacks that HTTP is vulnerable to that high-grade HTTPS is not.

Further, no different than standard HTTP / HTTPS web browsing, non-Torified traffic is vulnerable to all types of Certificate Authority, Domain Name System, and Border Gateway Protocol attacks. It is equally critical to discuss apt-transport-tor.

Debian claims that HTTPS is not for privacy. In August 2014, B_Meson filed a bug on this issue, and it was closed.

Debian claims that apt-secure is good enough for security. This boils down to: “By adding a key to apt’s keyring, you’re telling apt to trust everything signed by the key.” Debian cannot assure that these keys have not been compromised. In Ubuntu, there is still a 1024-bit master signing key from 2004 in the apt-key keychain that does not expire!

I’ve purchased “apt-transport-https.org” (not active) presuming that this will be the homepage for this initiative. The initiative includes documenting popular repos, grade their SSL/TLS, and shame organizations that are neglecting our security and privacy. Default security and privacy is a requirement.

Below is my initial list of popular repositories. Most fail. I am looking for feedback and advice on how we should document these problems and how we can best influence repository maintainers to care about modern security concerns.

Basically…

  • http is bad.
  • https is better for security.
  • tor+http is better for privacy.
  • tor+https is better for security and privacy.
  • http .onion is best for security and privacy.

Questions

  • Does apt-transport-https support configurations such as PFS, HSTS, HSTS Preload, or HPKP?
  • Can installing apt-transport-tor be configured to automatically replace known repos with an Onion replacement in sources.list?
  • Post-Snowden, why isn’t apt-transport-tor the default for all OS distributions?

Defaults

Debian OS:
http://httpredir.debian.org/
http://security.debian.org/

Ubuntu OS:
http://us.archive.ubuntu.com/
http://security.ubuntu.com/

Tails OS (using apt-transport-tor by default):
tor+http://deb.tails.boum.org/
tor+http://deb.torproject.org/
tor+http://ftp.us.debian.org/
tor+http://security.debian.org/

Subgraph OS (using apt-transport-tor by default):
tor+https://devrepo.subgraph.com/
tor+http://security.debian.org/
tor+http://httpredir.debian.org/

Kali OS:
http://http.kali.org/
http://security.kali.org/

Optional

Debian nor Ubuntu distinguishes HTTPS mirrors:

Debian OS mirrors (ideal):
http://m4dcywym6p6poxdm.onion/debian/ (Info: http://onionmirors63y7c.onion/)

Ubuntu PPAs:
http://ppa.launchpad.net/

Tor Project, Inc apps:
https://deb.torproject.org/

Notes

An excellent discussion concerning Tails. Tails is a Debian based client and has different threat models than Debian based servers.

I’ve started a Google Doc list grading mirrors.

Configurations

Here’s what an improved Ubuntu configuration might look like:

sudo apt-get install apt-transport-tor
sudo vim /etc/apt/sources.list
deb tor+https://ubuntu.wikimedia.org/ubuntu/ wily main restricted
deb tor+https://ubuntu.wikimedia.org/ubuntu/ wily-updates main restricted
deb tor+https://ubuntu.wikimedia.org/ubuntu/ wily universe
deb tor+https://ubuntu.wikimedia.org/ubuntu/ wily-updates universe
deb tor+https://ubuntu.wikimedia.org/ubuntu/ wily multiverse
deb tor+https://ubuntu.wikimedia.org/ubuntu/ wily-updates multiverse
deb tor+https://ubuntu.wikimedia.org/ubuntu/ wily-backports main restricted universe multiverse
deb tor+https://ubuntu.wikimedia.org/ubuntu/ wily-security main restricted
deb tor+https://ubuntu.wikimedia.org/ubuntu/ wily-security universe
deb tor+https://ubuntu.wikimedia.org/ubuntu/ wily-security multiverse

Debian example:

sudo apt-get install apt-transport-tor
sudo vim /etc/apt/sources.list
deb tor+http://m4dcywym6p6poxdm.onion/debian/ jessie main
deb tor+http://m4dcywym6p6poxdm.onion/debian/ jessie-updates main
deb tor+http://security.debian.org/ jessie-updates main