Animated Christmas Tree Written In Bash

2020-12-02T00:00:00+13:00

It’s Christmas OMG OMG OMG OMG 🎄 🎅 🤶
So here is your TerminalAddict.com gift for 2020 🎁🎁🎁

Prerequisites

You must have bash installed 😂

Assumptions

I’ll assume we have bash installed 😜

The code

cat << 'EOF' >> ~/xmas_tree.sh
#!/bin/bash

# tput is a command to manipulate the terminal, it can be used to change the color of text, apply effects, and generally brighten things up.
trap "tput reset; tput cnorm; exit" 2
clear
tput civis
lin=2
col=$(($(tput cols) / 2))
c=$((col-1))
est=$((c-2))
color=0

# Set the text color to green to write the tree
tput setaf 2; tput bold

# Write the tree
for ((i=1; i<40; i+=2))
{
    tput cup $lin $col
    for ((j=1; j<=i; j++))
    {
        echo -n \*
    }
    let lin++
    let col--
}

## Set the color to brown for the trunk
tput sgr0; tput setaf 130

# Write the Trunk in three lines
for ((i=1; i<=3; i++))
{
    tput cup $((lin++)) $c
    echo 'mWm'
}

# Write a greeting
tput setaf 93; tput bold
tput cup $lin $((c - 15)); echo Merry Christmas
tput cup $((lin + 1)) $((c - 15)); echo And a happy new year !!!!
let c++
k=1

# Configure lights and decorations
while true; do
    for ((i=1; i<=35; i++)) {
        # Turn off the lights
        [ $k -gt 1 ] && {
            tput setaf 2; tput bold
            tput cup ${line[$[k-1]$i]} ${column[$[k-1]$i]}; echo \*
            unset line[$[k-1]$i]; unset column[$[k-1]$i]  # Array cleanup
        }

        li=$((RANDOM % 9 + 10))
        start=$((c-li+2))
        co=$((RANDOM % (li-2) * 2 + 1 + start))
        tput setaf $color; tput bold   # Switch colors
        tput cup $li $co
        echo o
        line[$k$i]=$li
        column[$k$i]=$co
        color=$(((color+1)%8))
    }
    k=$((k % 2 + 1))
done


EOF

Now let’s run it

Execution

Set execution:

chmod 755 ~/xmas_tree.sh

Now run it:

~/xmas_tree.sh

The result

Check it out !!!

Happy Christmas! 🦌🦌🦌🦌🛷

Awstats Static Reports And Nginx

2020-09-24T00:00:00+12:00

So, I run a few websites 😉 and most of those websites (including this one) I have Google Analytics installed.
But server logs can provide different information. So I’m going to setup graphing and tracking my server access logs.

So my steps are going to be:

Install Awstats, along with some prerequisites for CPAN, and creating a password file.
Install CPAN The Comprehensive Perl Archive Network, along with the GeoIP plugin.
Write a bash script to create, update, and graph, some results (as well as a wee PHP script to make life easier).
Profit!

Prerequisites

I’m not going to talk you through setting up a Linux server, or Nginx, PHP-FPM (Fastcgi).

I will assume that you have a running server.
Your linux server can be any flavour you like, I prefer Debian and Ubuntu distros, and I prefer aptitude as a package manager; you can use whatever you like 🤗.

Assumptions

I’ll assume we store the Awstats DB in /var/lib/awstats.
I’ll assume we store the static reports in /var/cache/awstats.

Installing the packages

Here we go:
sudo aptitude install awstats libgeoip-dev build-essential openssl

Installing the GeoIP plugin

So first we need to setup CPAN:

$ sudo cpan
Loading internal logger. Log::Log4perl recommended for better logging

cpan shell -- CPAN exploration and modules installation (v2.28)
Enter 'h' for help.

cpan[1]>

Now, while in the CPAN shell, let’s install some stuff.
Firstly, CPAN itself, and then the GeoIP plugin.
cpan[1]> make install
cpan[1]> install Bundle::CPAN

That’ll get you up and running ready to install from cpan.
Now let’s install the GeoIP plugin:
cpan[1]> install Geo::IP

Setting up Nginx

So, a few prerequisites are needed for Nginx to behave properly.
Firstly, make sure you have PHP running using fastcgi (I’m not showing you how to do this in this doc).

Next, let’s make sure logging is separate for each website:
In my server { block I have the following lines for logging:

server_name terminaladdict.com;
access_log /var/log/nginx/terminaladdict.com-access.log;
error_log /var/log/nginx/terminaladdict.com-error.log;

I’m going to add some config to my server block specifically for awstats:

location ^~ /awstats {
	alias /var/cache/awstats/terminaladdict.com;
	location ~ \.php$ {
		fastcgi_split_path_info ^(.+\.php)(/.+)$;
		fastcgi_pass unix:/var/run/php/php7.3-fpm.sock;
		fastcgi_param SCRIPT_FILENAME $request_filename;
		include fastcgi_params;
	}
	auth_basic "Website Statistics";
	auth_basic_user_file /var/lib/awstats/.htpasswd;
}
location ^~ /awstats-icon {
	alias /usr/share/awstats/icon/;
}

Now reload nginx systemctl reload nginx.service

Password protect our stats

Let’s not go public with our stats 😆
In the Nginx config above I provided for password protection, so let’s create that .htpasswd file:

printf "Hi_my_name_is:`openssl passwd -apr1`\n" >> /var/lib/awstats/.htpasswd
Password: *********
Verifying - Password: *********

Setting up Awstats

Right, so for whatever reason, awstats installs a cron job that doesn’t seem to work.
So let’s turn it off:
echo "" > /etc/cron.d/awstats
phew 😆

Now let’s create an awstats config for our website.
Awstats expects the config to be named “awstats.website.ext.conf”.
Something like:

awstats.terminaladdict.com.conf
awstats.example.co.nz.conf
awstats.subdomain.example.org.nz.conf

I’m creating a config for terminaladdict.com, so here we have it:

LogFile="/var/log/nginx/terminaladdict.com-access.log"
SiteDomain="terminaladdict.com"
HostAliases="www.terminaladdict.com"
DirIcons="/awstats-icon"
DirData="/var/lib/awstats"
DNSLookup = 1
LoadPlugin="geoip GEOIP_STANDARD /usr/share/GeoIP/GeoIP.dat"

Here’s a warning ! If your website is super busy, probably disable DNSLookup, by setting it to 0
DNSLooup = 1 will slow things down A LOT

The bash script

While I was setting this up I read many articles about configuring nginx to using cgi-bin.php with fastcgi.
For me, it just didn’t work, and I got bored trying to figure out why it didn’t work!

So I wrote my own little bash script to update awstats db, generate static reports, and build an index.php for some of the directories, just for ease of use.

Here is my script:

#!/bin/bash

AWSTATS=/usr/lib/cgi-bin/awstats.pl
DIR=/var/cache/awstats
YEAR=`date +%Y`
MONTH=`date +%m`
PHPFILE="
<!DOCTYPE html>
<html lang='en'>
<head>
    <meta charset='utf-8'>
    <meta name='viewport' content='width=device-width, initial-scale=1, shrink-to-fit=no'>
    <title>Awstats</title>
    <link rel='stylesheet' href='https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/css/bootstrap.min.css'>
    <meta name='description' content='Awstats for websites hosted on Net Enterprises Ltd' />
</head>
<body>

<div class='container-fluid pt-5'>
    <div class='container'>
        <div class='row'>
            <div class='col-md-12'>
                <h3>Stuff found in this directory</h3>
                <ul class='list-unstyled'>
<?php
\$dir = '.';
\$files = scandir(\$dir);
\$excludes = Array('.', '..', 'index.php');

foreach(\$files as \$file) {
    if(!in_array(\$file, \$excludes)) {
        echo '                  <li class=\'p-3\'><a data-toggle=\'tooltip\' data-original-title=\''.\$file.'\' data-placement=\'bottom\' href=\''.\$file.'\' class=\'btn btn-primary btn-lg\'>'.\$file.'</a></li>' . PHP_EOL;
    }
}
?>
                </ul>
            </div>
        </div>
    </div>
</div>
<script src='https://code.jquery.com/jquery-3.5.1.slim.min.js'></script>
<script src='https://unpkg.com/@popperjs/core@2'></script>
<script src='https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/js/bootstrap.min.js'></script>
</body>
</html>
";

for c in `/bin/ls -1 /etc/awstats/awstats.*.conf 2>/dev/null | /bin/sed 's/^\/etc\/awstats\/awstats\.\(.*\)\.conf/\1/'` `[ -f /etc/awstats/awstats.conf ]`
do
    mkdir -p $DIR/$c/$YEAR/$MONTH/
    for d in $DIR $DIR/$c $DIR/$c/$YEAR
    do
        echo "$PHPFILE" > $d/index.php
    done
    $AWSTATS -config=$c -update

    # report for the year
    $AWSTATS -config=$c -output -month=all -staticlinks > $DIR/$c/$YEAR/awstats.yearly.report.$c.html
    ln -sf $DIR/$c/$YEAR/awstats.yearly.report.$c.html $DIR/$c/$YEAR/1-start-here.html

    # report the for the current month
    $AWSTATS -config=$c -output -staticlinks  > $DIR/$c/$YEAR/$MONTH/awstats.$c.html
    ln -sf $DIR/$c/$YEAR/$MONTH/awstats.$c.html $DIR/$c/$YEAR/$MONTH/index.html

    for stat in alldomains allhosts lasthosts unknownip alllogins lastlogins allrobots lastrobots urldetail urlentry urlexit browserdetail osdetail unknownbrowser unknownos refererse refererpages keyphrases keywords errors404 downloads
    do
        $AWSTATS -config=$c -output=$stat -month=all -staticlinks > $DIR/$c/$YEAR/awstats.$c.$stat.html
        $AWSTATS -config=$c -output=$stat -staticlinks > $DIR/$c/$YEAR/$MONTH/awstats.$c.$stat.html
    done
done

What the script does is:

Set up some vairable, including a variable named PHPFILE, that creates a PHP file for some directories.
in a for loop, for each awstats.{websitename}.conf do
- make a directory if needed (mkdir -p)
- a new for loop, that creates a PHP file in the root directory, and the $YEAR directory
- Updates the stats $AWSTATS -config=$c -update
- create some soft links for ease of use
- a nested for loop that creates reports for the year, and the month
- 1). $AWSTATS -config=$c -output=$stat -month=all -staticlinks > $DIR/$c/$YEAR/awstats.$c.$stat.html
- 2). $AWSTATS -config=$c -output=$stat -staticlinks > $DIR/$c/$YEAR/$MONTH/awstats.$c.$stat.html
profit!

And here is the result:

Automating

And of course, chuck it in cron so it updates regularly.
I run mine at 17 minutes passed the hour, because - well I dunno - 17 seemed like a fun number to use 🤪.
I stored the script in /usr/local/bin and called it awstats.update_and_build.sh

# awstats
17 *    * * *   root    /usr/local/bin/awstats.update_and_build.sh > /dev/null 2>&1

Happy stats collecting !

Graphing Speedtest Results

2020-09-21T00:00:00+12:00

I can be described as a network engineer, sometimes, and sometimes a system admin. As such, keeping track of how customers internet is working is reasonably important.
Most customers use Speedtest.net as a way of testing their internet connection, so I figured I would clobber together a script that keeps a bit of history, and make some pretty graphs 😅.

So my steps are going to be:

Install rrdtool, Gawk, and grep
Install Ookla speedtest cli.
Write a bash script to create, update, and graph, some results
Profit!

Prerequisites

I’m not going to talk you through setting up a Linux server.

I’m also not going to talk you through setting up a web server .. sort that out yourself 😜.

You NEED unlimited data on your internet!!!!
Let me say that again .. YOU NEED UNLIMITED DATA ON YOUR INTERNET !!!

This script will thrash your internet usage!

Installing the packages, and setting up directories

Right, let’s start with installing rrdtool, Gawk, and grep
sudo aptitude install rrdtool gawk grep

That’s pretty easy !

Install Ookla Speedtest cli

So you can head over to the Ookla Speedtest cli website and download the appropriate package. In my case it’s the Linux x86_64 package, which is at version 1.0.0.
wget https://bintray.com/ookla/download/download_file?file_path=ookla-speedtest-1.0.0-x86_64-linux.tgz -O /tmp/speedtest.tgz

Now let’s unpack it:
tar xvfz /tmp/speedtest.tgz -C /tmp

And put it in a usable spot on the OS:
sudo mv /tmp/speedtest /usr/local/bin/speedtest

Running Speedtest cli

The first time you run speedtest you will be asked to agree to a license:

/usr/local/bin/speedtest
==============================================================================

You may only use this Speedtest software and information generated
from it for personal, non-commercial use, through a command line
interface on a personal computer. Your use of this software is subject
to the End User License Agreement, Terms of Use and Privacy Policy at
these URLs:

        https://www.speedtest.net/about/eula
        https://www.speedtest.net/about/terms
        https://www.speedtest.net/about/privacy

==============================================================================

Do you accept the license? [type YES to accept]:

Now you’re ready to go!

You can list out the nearest servers like so:

/usr/local/bin/speedtest -L
Closest servers:

    ID  Name                           Location             Country
==============================================================================
  5469  MyRepublic Limited             Auckland             New Zealand
  5749  Vocusgroup NZ                  Auckland             New Zealand
 11327  Spark New Zealand              Auckland             New Zealand
  2720  WorldNet                       Auckland             New Zealand
  4953  Vodafone New Zealand           Auckland             New Zealand
   721  WorldxChange Comm              Auckland             New Zealand
 13676  speedtest.nzpbx.com            Auckland             New Zealand
 12932  Feenix Communications Limited  Auckland             New Zealand
 16805  Nova Energy                    AUCKLAND             New Zealand
 25477  Devoli                         Auckland             New Zealand

This particular customer I am testing from is connected through Vocus.
So I’m going to use Vocus to test against.

/usr/local/bin/speedtest -s 5749

   Speedtest by Ookla

     Server: Vocusgroup NZ - Auckland (id = 5749)
        ISP: Vocus Communications
    Latency:     5.10 ms   (0.37 ms jitter)
   Download:   844.16 Mbps (data used: 808.7 MB)
     Upload:   548.05 Mbps (data used: 267.4 MB)
Packet Loss:     0.0%
 Result URL: https://www.speedtest.net/result/c/1ff5533b-b0de-42c5-bd14-24c299cfc249

Sweet !

Create a directory

Ok, so now let’s create a directory to store history, and images.
sudo mkdir /var/www/html/rrd
sudo chown paul:paul /var/www/html/rrd

Note: I have changed the owner of the directory to the user ‘paul’. Change to whatever user you’re going to be running this script as.

The RRD Script

Ok, so here’s where the magic happens. So let’s skip straight to the script:

#!/bin/bash
# generate a speedtest result
# 5749 server is Vocus
/usr/local/bin/speedtest -s 5749 -p no > /tmp/speedtest.txt 2>/dev/null

TRAF=/var/www/html/rrd

        case $1 in (create)
                /usr/bin/rrdtool create $TRAF/upload.rrd -s 60 \
                DS:upload:GAUGE:600:0:U \
                RRA:AVERAGE:0.5:1:4320 \
                RRA:AVERAGE:0.5:1440:3 \
                RRA:MIN:0.5:1440:3 \
                RRA:MAX:0.5:1440:3
                /usr/bin/rrdtool create $TRAF/download.rrd -s 60 \
                DS:download:GAUGE:600:0:U \
                RRA:AVERAGE:0.5:1:4320 \
                RRA:AVERAGE:0.5:1440:3 \
                RRA:MIN:0.5:1440:3 \
                RRA:MAX:0.5:1440:3
                /usr/bin/rrdtool create $TRAF/echoreply.rrd -s 60 \
                DS:echoreply:GAUGE:600:0:U \
                RRA:AVERAGE:0.5:1:4320 \
                RRA:AVERAGE:0.5:1440:3 \
                RRA:MIN:0.5:1440:3 \
                RRA:MAX:0.5:1440:3
                ;;
        (update)
                /usr/bin/rrdtool update $TRAF/upload.rrd N:`cat /tmp/speedtest.txt | grep Upload | awk '{print $3}'`
                /usr/bin/rrdtool update $TRAF/download.rrd N:`cat /tmp/speedtest.txt | grep Download | awk '{print $3}'`
                /usr/bin/rrdtool update $TRAF/echoreply.rrd N:`cat /tmp/speedtest.txt | grep Latency | awk '{print $2}'`
                ;;
        (graph)
                /usr/bin/rrdtool graph $TRAF/upload.png \
                --start "-3day" \
                -c "BACK#000000" \
                -c "SHADEA#000000" \
                -c "SHADEB#000000" \
                -c "FONT#DDDDDD" \
                -c "CANVAS#202020" \
                -c "GRID#666666" \
                -c "MGRID#AAAAAA" \
                -c "FRAME#202020" \
                -c "ARROW#FFFFFF" \
                -u 1.1 -l 0 -v "Upload" -w 1100 -h 250 -t "Upload Speed - `/bin/date +%A", "%d" "%B" "%Y`" \
                DEF:upload=$TRAF/upload.rrd:upload:AVERAGE \
                AREA:upload\#FFFF00:"Upload speed (Mbit/s)" \
                GPRINT:upload:MIN:"Min\: %3.2lf " \
                GPRINT:upload:MAX:"Max\: %3.2lf" \
                GPRINT:upload:LAST:"Current\: %3.2lf\j" \
                COMMENT:"\\n"

                /usr/bin/rrdtool graph $TRAF/download.png \
                --start "-3day" \
                -c "BACK#000000" \
                -c "SHADEA#000000" \
                -c "SHADEB#000000" \
                -c "FONT#DDDDDD" \
                -c "CANVAS#202020" \
                -c "GRID#666666" \
                -c "MGRID#AAAAAA" \
                -c "FRAME#202020" \
                -c "ARROW#FFFFFF" \
                -u 1.1 -l 0 -v "Download" -w 1100 -h 250 -t "Download Speed - `/bin/date +%A", "%d" "%B" "%Y`" \
                DEF:download=$TRAF/download.rrd:download:AVERAGE \
                AREA:download\#00FF00:"Download speed (Mbit/s)" \
                GPRINT:download:MIN:"Min\: %3.2lf " \
                GPRINT:download:MAX:"Max\: %3.2lf" \
                GPRINT:download:LAST:"Current\: %3.2lf\j" \
                COMMENT:"\\n"

                /usr/bin/rrdtool graph $TRAF/echoreply.png \
                --start "-3day" \
                -c "BACK#000000" \
                -c "SHADEA#000000" \
                -c "SHADEB#000000" \
                -c "FONT#DDDDDD" \
                -c "CANVAS#202020" \
                -c "GRID#666666" \
                -c "MGRID#AAAAAA" \
                -c "FRAME#202020" \
                -c "ARROW#FFFFFF" \
                -u 1.1 -l 0 -v "Ping" -w 1100 -h 250 -t "Ping Response - `/bin/date +%A", "%d" "%B" "%Y`" \
                DEF:echoreply=$TRAF/echoreply.rrd:echoreply:AVERAGE \
                AREA:echoreply\#FF0000:"Ping Response (ms)" \
                GPRINT:echoreply:MIN:"Min\: %3.2lf " \
                GPRINT:echoreply:MAX:"Max\: %3.2lf" \
                GPRINT:echoreply:LAST:"Current\: %3.2lf\j" \
                COMMENT:"\\n"
                ;;

        (*)
                echo "Invalid option.";;
        esac

Let’s do some ‘splaining

So firstly, copy everything, save it in a file (I called mine speedtest_rrd.sh), and chmod appropriately.
Let me make an assumption that all your binary or executable files are in ~/bin.

chmod 755 ~/bin/speedtest_rrd.sh

The script can take one argument ( $1 ).

Argument 1: (create)
This creates the rrd files to store history in, and creates some rrd info that we are going to display.

Argument 2: (update)
This updates the rrd database with new values.

Argument 3: (graph)
This draws the pretty graphs !

You may have noticed right at the very beginning of the script this line:
/usr/local/bin/speedtest -s 5749 -p no > /tmp/speedtest.txt 2>/dev/null

This runs the speedtest cli, and saves the output into your /tmp directory.

Running the script

Now let’s run it, and schedule it, then look at the results.

Running the script manually

Firstly we need to create the rrd database:
~/bin/speedtest_rrd.sh create

result:

ls -l /var/www/html/rrd/*.rrd
-rw-r--r-- 1 paul paul 35840 Sep 20 22:20 /var/www/html/rrd/download.rrd
-rw-r--r-- 1 paul paul 35840 Sep 20 22:20 /var/www/html/rrd/echoreply.rrd
-rw-r--r-- 1 paul paul 35840 Sep 20 22:20 /var/www/html/rrd/upload.rrd

simple !

Now let’s do a test:
~/bin/speedtest_rrd.sh update

and now, let’s create a graph:
~/bin/speedtest_rrd.sh graph

ls -l /var/www/html/rrd/
total 188
-rw-rw-r-- 1 paul paul 28167 Sep 20 22:20 download.png
-rw-r--r-- 1 paul paul 35840 Sep 20 22:20 download.rrd
-rw-rw-r-- 1 paul paul 27113 Sep 20 22:20 echoreply.png
-rw-r--r-- 1 paul paul 35840 Sep 20 22:20 echoreply.rrd
-rw-rw-r-- 1 paul paul 21572 Sep 20 22:20 upload.png
-rw-r--r-- 1 paul paul 35840 Sep 20 22:20 upload.rrd

Alright ! we have some data !

Scheduling the script

Ok, so now we have some data, let’s start auto running the data collection.

Remember me saying at the beginning you need unlimited data on your internet?

Let me say that again .. YOU NEED UNLIMITED DATA ON YOUR INTERNET !!!
This script will thrash your internet usage!

So with that warning out of the way, let’s chuck it in cron:

echo "*/5 *   * * *   paul /home/paul/bin/speedtest_rrd.sh update > /dev/null 2>&1 && /home/paul/bin/speedtest_rrd.sh graph > /dev/null 2>&1" >> /etc/crontab

note: I am running this script as the user ‘paul’. Change to whatever user you are running this script as.

The result

That’s it! we’re running. Let that script chug away for a few days, and you’ll get a result that looks like this:

Happy speedtesting !

Streaming A Zoneminder Monitor To A Web Site

2020-09-11T00:00:00+12:00

So I run a CCTV system based on the excellent open source Zoneminder, running on a pretty entry level ex-lease PC I bought from PBTech.

What I want to do is embed one of the cameras into a web site.

So my steps are going to be:

Setup a Zoneminder user for public access.
Write a small PHP proxy script.
Profit!

Prerequisites

I’m not going to talk you through setting up a Debian server.
Nor am I going to talk you through setting up a Zoneminder server on a Debian server.

I will assume that you have these things running.
I’ll also assume that you have the Zoneminder console running from a virtual sub directory in Apache (in my case /zm/ ).

But just in case, here’s a quick reminder of the Apache config:

# Remember to enable cgi mod (i.e. "a2enmod cgi").
ScriptAlias /zm/cgi-bin "/usr/lib/zoneminder/cgi-bin"
<Directory "/usr/lib/zoneminder/cgi-bin">
    Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
    AllowOverride All
    Require all granted
</Directory>


# Order matters. This alias must come first.
Alias /zm/cache /var/cache/zoneminder/cache
<Directory /var/cache/zoneminder/cache>
    Options -Indexes +FollowSymLinks
    AllowOverride None
    <IfModule mod_authz_core.c>
        # Apache 2.4
        Require all granted
    </IfModule>
    <IfModule !mod_authz_core.c>
        # Apache 2.2
        Order deny,allow
        Allow from all
    </IfModule>
</Directory>

Alias /zm /usr/share/zoneminder/www
<Directory /usr/share/zoneminder/www>
  Options -Indexes +FollowSymLinks
  <IfModule mod_dir.c>
    DirectoryIndex index.php
  </IfModule>
</Directory>

# For better visibility, the following directives have been migrated from the
# default .htaccess files included with the CakePHP project.
# Parameters not set here are inherited from the parent directive above.
<Directory "/usr/share/zoneminder/www/api">
   RewriteEngine on
   RewriteRule ^$ app/webroot/ [L]
   RewriteRule (.*) app/webroot/$1 [L]
   RewriteBase /zm/api
</Directory>

<Directory "/usr/share/zoneminder/www/api/app">
   RewriteEngine on
   RewriteRule ^$ webroot/ [L]
   RewriteRule (.*) webroot/$1 [L]
   RewriteBase /zm/api
</Directory>

<Directory "/usr/share/zoneminder/www/api/app/webroot">
    RewriteEngine On
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteRule ^ index.php [L]
    RewriteBase /zm/api
</Directory>

Zoneminder read only user

Right, let’s start with a Zoneminder “read only” user.

In your Zoneminder console, head to Options->User, and smash that “ADD NEW USER” button.

Pretty straight forward:

Username = YOURPUBLICUSER
Password = YOURPUBLICPASS
- Confirm Password = YOURPUBLICPASS
Language
Enabled = yes
Stream = View
Events = None
Control = None
Monitors = View
Groups = None
System = None
Max bandwidth = whatever you like
Restricted monitors = Select the Cameras you want available publicly.
API Enabled = No

Save that bad boy.
That’s step 1 done 😊

A PHP Proxy script

So at this point you could just jam an image in any web site, using a specially crafted URL:
http://YOUR-ZONEMINDER-CCTV-URL/zm/cgi-bin/nph-zms?monitor=1&user=YOURPUBLICUSER&pass=YOURPUBLICPASS

But, that kinda exposes, well, everything ! Your CCTV url, your readonly user and pass. I dunno, it just doesn’t seem right to me 🤣
So, I’m going to craft a small PHP proxy script which will hide this info from public view.

<?php
// storing this here just for future reference
$streamurl =  "http://YOUR-ZONEMINDER-CCTV-URL/zm/cgi-bin/nph-zms?monitor=5&user=YOURPUBLICUSER&pass=YOURPUBLICPASS";

$host = "YOUR-ZONEMINDER-CCTV-URL";
$zmMonitor = "1";
$zmUser = "YOURPUBLICUSER";
$zmPass = "YOURPUBLICPASS";
$requestURI = "/zm/cgi-bin/nph-zms";

$getARGS = $requestURI."?monitor=".$zmMonitor."&user=".$zmUser."&pass=".$zmPass;

set_time_limit(0);
$fp = fsockopen ($host, 80, $errno, $errstr, 30);
if (!$fp) {
    echo "$errstr ($errno)<br>\n";
} else {
    fputs ($fp, "GET ".$getARGS." HTTP/1.0\r\n\r\n");
    while ($str = trim(fgets($fp, 4096)))
       header($str);
    fpassthru($fp);
    fclose($fp);
}
?>

That’s it; save that PHP script as .. I dunno .. zoneminder_stream.php ?

Embed in HTML

Here’s the fun bit .. simple HTML !
<img src="/path/to/zoneminder_stream.php" alt="my stream" />

and here is the working result:

*note: this is a screenshot of my working camera

Lets Encrypt Setup On Debian

2020-02-13T00:00:00+13:00

So in a few of my posts I’ve mentioned Let’s Encrypt as a method of getting SSL Certificates for free.
In this post I’ll explain how I set up Nginx to use Let’s Encrypt SSL certs.

Initial install and setup

I decide a while ago that Dehydrated would be the script I would use to manage my SSL certs. And conveniently for me there is a Debian package. So let’s install it.

aptitude install dehydrated

This creates a config directory in /etc/dehydrated and stores all your certs in /var/lib/dehydrated/certs.
Great, let’s config it.

#############################################################
# This is the main config file for dehydrated               #
#                                                           #
# This is the default configuration for the Debian package. #
# To see a more comprehensive example, see                  #
# /usr/share/doc/dehydrated/examples/config                 #
#                                                           #
# For details please read:                                  #
# /usr/share/doc/dehydrated/README.Debian                   #
#############################################################
CONFIG_D=/etc/dehydrated/conf.d
BASEDIR=/var/lib/dehydrated
WELLKNOWN="/var/www/dehydrated"
DOMAINS_TXT="/etc/dehydrated/domains.txt"
CONTACT_EMAIL=paul@domain.com
HOOK=/etc/dehydrated/hook.sh

Ok, so there’s a few things in this config we’re going to have to create

the directory /var/www/dehydrated
the file /etc/dehydrated/domains.txt
the hook file /etc/dehydrated/hook.sh

So let’s do that:

mkdir /var/www/dehydrated && chown www-data:www-data /var/www/dehydrated
Pretty simple so far.

The domains file

terminaladdict.com www.terminaladdict.com myother-subdomain.terminaladdict.com

Let’s Encrypt creates one certificate “per line” i.e. it creates one certificate that covers the domain terminaladdict.com, and the sub domains.
The sub domains must all exist in DNS, and all exist in Nginx.

The Hook File

I got this hook file from Antoine Aflalo
Here it is:

deploy_challenge() {
     local DOMAIN="${1}" TOKEN_FILENAME="${2}" TOKEN_VALUE="${3}"
}
clean_challenge() {
 local DOMAIN="${1}" TOKEN_FILENAME="${2}" TOKEN_VALUE="${3}"
}
deploy_cert() {
 local DOMAIN="${1}" KEYFILE="${2}" CERTFILE="${3}" FULLCHAINFILE="${4}" CHAINFILE="${5}" TIMESTAMP="${6}"

 systemctl reload nginx
}
unchanged_cert() {
 local DOMAIN="${1}" KEYFILE="${2}" CERTFILE="${3}" FULLCHAINFILE="${4}" CHAINFILE="${5}"
}
invalid_challenge() {
 local DOMAIN="${1}" RESPONSE="${2}"
}
request_failure() {
 local STATUSCODE="${1}" REASON="${2}" REQTYPE="${3}"
}
exit_hook() {
 :
}
HANDLER="$1"; shift
if [[ "${HANDLER}" =~ ^(deploy_challenge|clean_challenge|deploy_cert|unchanged_cert|invalid_challenge|request_failure|exit_hook)$ ]]; then
 "$HANDLER" "$@"
fi

Nginx Config

Setting up Nginx is pretty straight forward, you need a “.well-known” directive:

server {
    listen 80;
    listen [::]:80;

    location /.well-known/acme-challenge {
        alias /var/www/dehydrated;
    }
    root /var/www/html/terminaladdict.com;

    server_name terminaladdict.com www.terminaladdict.com;
}

that’s our entire config for port 80 (HTTP).

For the SSL port 443 (HTTPS) config:

server {
	listen 443 ssl http2;
	listen [::]:443 ssl http2;

    location /.well-known/acme-challenge {
      alias /var/www/dehydrated;
    }
    ssl_certificate /var/lib/dehydrated/certs/terminaladdict.com/fullchain.pem;
    ssl_certificate_key /var/lib/dehydrated/certs/terminaladdict.com/privkey.pem;

	# enables all versions of TLS, but not SSLv2 or 3 which are weak and now deprecated.
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

    # disables all weak ciphers
    ssl_ciphers 'AES128+EECDH:AES128+EDH';

    ssl_prefer_server_ciphers on;

    root /var/www/html/terminaladdict.com;
    client_max_body_size 100m;

    index index.html;

    server_name terminaladdict.com;
    access_log /var/log/nginx/terminaladdict.com-access.log;
    error_log /var/log/nginx/terminaladdict.com-error.log;

    location / {
        try_files $uri $uri/ index.html;
        add_header Service-Worker-Allowed /;
    }
    error_page 404 =404 /404.html;

    location /manifest.json {
        default_type application/x-web-app-manifest+json;
    }

    location ~ /\.ht {
        deny all;
    }
    # ESSENTIAL : no favicon logs
    location = /favicon.ico {
        log_not_found off;
        access_log off;
    }
    # ESSENTIAL : robots.txt
    location = /robots.txt {
        allow all;
        log_not_found off;
        access_log off;
    }
    location ~* sw\.js$ {
        expires -1;
    }
    # PERFORMANCE : Set expires headers for static files and turn off logging.
    location ~* ^.+\.(js|css|swf|xml|txt|ogg|ogv|svg|svgz|eot|otf|woff|mp4|ttf|rss|atom|jpg|jpeg|gif|png|ico|zip|tgz|gz|rar|bz2|doc|xls|exe|ppt|tar|mid|midi|wav|bmp|rtf)$ {
            access_log off; log_not_found off; expires 30d;
    }
}

You won’t be able to restart Nginx yet, as you don’t have the certs.
What I do is comment out the two lines that point to the certs, reload Nginx, then uncomment them once I have the certs.

Getting the cert

Here we go:
dehydrated -fc -c

You should see output like this:

Processing terminaladdict.com with alternative names: www.terminaladdict.com
 + Checking domain name(s) of existing cert... unchanged.
 + Checking expire date of existing cert...
 + Valid till Mar 12 12:05:56 2020 GMT Certificate will expire
(Less than 30 days). Renewing!
 + Signing domains...
 + Generating private key...
 + Generating signing request...
 + Requesting new certificate order from CA...
 + Received 2 authorizations URLs from the CA
 + Handling authorization for terminaladdict.com
 + Handling authorization for www.terminaladdict.com
 + 2 pending challenge(s)
 + Deploying challenge tokens...
 + Responding to challenge for terminaladdict.com authorization...
 + Challenge is valid!
 + Responding to challenge for www.terminaladdict.com authorization...
 + Challenge is valid!
 + Cleaning challenge tokens...
 + Requesting certificate...
 + Checking certificate...
 + Done!
 + Creating fullchain.pem...
 + Done!

That’s it! go and load your website in a browser and check out your new SSL certificate!

Automating

Hello cron:

# lets encrypt
5 2     * * 6   root    /usr/bin/dehydrated -fc -c > /dev/null 2>&1

Dns Over Https Or (doh) Bind9 And Nginx

2019-09-13T00:00:00+12:00

So in a litte while Google Chrome is going to enable DNS over HTTPs, and Firefox has already enabled it, by default!
So I think to myself “myself, do you want cloudflare watching all your DNS queries?” - nope is the the answer!
Right, so I’ll build my own DNS over HTTPs (DoH) server.

So what I’m going to do is:

Setup nginx to act as a front end proxy.
Build and install a go package called doh-server
Check that it all works 👨‍💻
Configure my browser

This doc assumes you have a working DNS server (I use bind9), and you can sort your own SSL certs for nginx.
It also assumes you know your way around a command line 🤪

Set up nginx

So I’m not going to talk you through how to install nginx, or get it running, that is beyond the scope of this doc.
I’m also not going to tell you how to setup Let’s Encrypt, that is also outside the scope of this doc.

So let’s get straight into the config:

upstream dns-backend {
    server 127.0.0.1:8053;
    keepalive 30;
}
server {
    listen 80;
    listen [::]:80;

    location /.well-known/acme-challenge {
        alias /var/www/dehydrated;
    }
    root /var/www/html;

    server_name ns1.example.com;
    return 301 https://$host;
}

server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;

    location /.well-known/acme-challenge {
      alias /var/www/dehydrated;
    }
    ssl_certificate /etc/dehydrated/certs/ns1.example.com/fullchain.pem;
    ssl_certificate_key /etc/dehydrated/certs/ns1.example.com/privkey.pem;

    # enables all versions of TLS, but not SSLv2 or 3 which are weak and now deprecated.
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

    # disables all weak ciphers
    ssl_ciphers 'AES128+EECDH:AES128+EDH';

    ssl_prefer_server_ciphers on;

    root /var/www/html;

    index index.html index.htm index.nginx-debian.html;

    server_name ns1.example.com;
    access_log /var/log/nginx/ns1.example.com-access.log;
    error_log /var/log/nginx/ns1.example.com-error.log;

    location /dns-query {
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_set_header X-NginX-Proxy true;
        proxy_set_header Connection "";
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_redirect off;
        proxy_set_header        X-Forwarded-Proto $scheme;
        proxy_read_timeout 86400;
        proxy_pass http://dns-backend/dns-query;
    }

}

So let me talk you through the config:
I start by creating a config that points to our, not yet installed, doh-server. That server is going to be running on port 8053:

upstream dns-backend {
    server 127.0.0.1:8053;
    keepalive 30;
}

Next I’ve got a server block for port 80 (non-secure / non https).
It has some config for my Let’s Encrypt SSL cert bot (dehydrated), and redirects everthing to the SSL version of the website (return 301):

server {
    listen 80;
    listen [::]:80;

    location /.well-known/acme-challenge {
        alias /var/www/dehydrated;
    }
    root /var/www/html;

    server_name ns1.example.com;
    return 301 https://$host;
}

and lastly, the SSL version config, includng the reverse proxy bit which I will repeat here (because it is the important bit):

location /dns-query {
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_set_header X-NginX-Proxy true;
    proxy_set_header Connection "";
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_redirect off;
    proxy_set_header        X-Forwarded-Proto $scheme;
    proxy_read_timeout 86400;
    proxy_pass http://dns-backend/dns-query;
}

So assuming that you have your SSL certs sorted, and your nginx config is all good you should be able to start nginx 🥰

Install doh-server

so you’ll need some build packages, so let’s install some build tools:
aptitude install curl software-properties-common build-essential git

And you’re going to need to get your golang environment setup:
Install Go on Debian Buster

now let’s install the DoH server:

git clone git@github.com:m13253/dns-over-https.git
cd dns-over-https/
make
make install

Now let’s config the DoH server:
The DoH server has a config in /etc/dns-over-https/doh-server.conf
You want to change the upstream to use 127.0.0.1:53 and that’s about it, here’s what I’ve got:

listen = [
    "127.0.0.1:8053",
    "[::1]:8053"
]
local_addr = ""
cert = ""
key = ""
path = "/dns-query"
upstream = [
	"127.0.0.1:53"
]
timeout = 10
tries = 3
tcp_only = false
verbose = false
log_guessed_client_ip = false

Once you’ve got that saved you can restart the DoH server with systemctl restart doh-server

Testing the install

So let’s recap:
We have:

Configured nginx and got it running nicely, forwarding DoH requests
Installed a DoH server
We already had a DNS resolver installed and running, right? 🧐

Now let’s do some testing.

The DoH server returns JSON, so you can just test in your browser:
https://ns1.example.com/dns-query?name=example.org&type=A

Or, since you’re already working in a terminal (and this website is called Terminal Addict 😇)

curl -s "https://ns1.example.com/dns-query?name=example.org&type=A" | python -m json.tool

You should get a JSON response like the following:

{
    "AD": true,
    "Answer": [
        {
            "Expires": "Thu, 12 Sep 2019 22:59:07 UTC",
            "TTL": 7090,
            "data": "93.184.216.34",
            "name": "example.org.",
            "type": 1
        },
        {
            "Expires": "Thu, 12 Sep 2019 22:59:07 UTC",
            "TTL": 7090,
            "data": "A 8 2 86400 20190921055451 20190831145703 31036 example.org. r4fWzdiYS7Px0qXX+7cHtPsj2a3lFhmeM4OXvOsDd2WorpP3Na/FupUFozdp2ao7xyguW+tZSJEI01dKuMt5MDfmJR4cN2n+IiRMvLvQuG60SnQNRBmOTTwca79GNTv5To8rxU5kSixH1liyho2/c/hvIqZKajHs6Bxr460A/RM=",
            "name": "example.org.",
            "type": 46
        }
    ],
    "Authority": [
        {
            "Expires": "Thu, 12 Sep 2019 21:18:35 UTC",
            "TTL": 1058,
            "data": "l.root-servers.net.",
            "name": ".",
            "type": 2
        },
        {
            "Expires": "Thu, 12 Sep 2019 21:18:35 UTC",
            "TTL": 1058,
            "data": "j.root-servers.net.",
            "name": ".",
            "type": 2
        },
        {
            "Expires": "Thu, 12 Sep 2019 21:18:35 UTC",
            "TTL": 1058,
            "data": "h.root-servers.net.",
            "name": ".",
            "type": 2
        },
        {
            "Expires": "Thu, 12 Sep 2019 21:18:35 UTC",
            "TTL": 1058,
            "data": "e.root-servers.net.",
            "name": ".",
            "type": 2
        },
        {
            "Expires": "Thu, 12 Sep 2019 21:18:35 UTC",
            "TTL": 1058,
            "data": "f.root-servers.net.",
            "name": ".",
            "type": 2
        },
        {
            "Expires": "Thu, 12 Sep 2019 21:18:35 UTC",
            "TTL": 1058,
            "data": "g.root-servers.net.",
            "name": ".",
            "type": 2
        },
        {
            "Expires": "Thu, 12 Sep 2019 21:18:35 UTC",
            "TTL": 1058,
            "data": "k.root-servers.net.",
            "name": ".",
            "type": 2
        },
        {
            "Expires": "Thu, 12 Sep 2019 21:18:35 UTC",
            "TTL": 1058,
            "data": "b.root-servers.net.",
            "name": ".",
            "type": 2
        },
        {
            "Expires": "Thu, 12 Sep 2019 21:18:35 UTC",
            "TTL": 1058,
            "data": "d.root-servers.net.",
            "name": ".",
            "type": 2
        },
        {
            "Expires": "Thu, 12 Sep 2019 21:18:35 UTC",
            "TTL": 1058,
            "data": "i.root-servers.net.",
            "name": ".",
            "type": 2
        },
        {
            "Expires": "Thu, 12 Sep 2019 21:18:35 UTC",
            "TTL": 1058,
            "data": "a.root-servers.net.",
            "name": ".",
            "type": 2
        },
        {
            "Expires": "Thu, 12 Sep 2019 21:18:35 UTC",
            "TTL": 1058,
            "data": "m.root-servers.net.",
            "name": ".",
            "type": 2
        },
        {
            "Expires": "Thu, 12 Sep 2019 21:18:35 UTC",
            "TTL": 1058,
            "data": "c.root-servers.net.",
            "name": ".",
            "type": 2
        },
        {
            "Expires": "Thu, 12 Sep 2019 21:18:35 UTC",
            "TTL": 1058,
            "data": "NS 8 0 518400 20190925170000 20190912160000 59944 . E7Vt9bijsTA8J2m5mcI9mc7uyk2PAbjtuDS4zGFaDVpVP4UyFFPYrttCA5CrXFurVN+Qf0nzB31pLzPfrgnY2xqe2S0Pm7Qq+t7rCp/Q1mNp3JbZG4fYxRXoz9uz4zSCaXDJ9WP4zrWrYd++ZGuUs4DXawg9qo0QhX5KRnXehDM9WkjY3JM61tw8CN5KS1ODov7Nw5qZI2uJ69npPnElm163cePQgZTkoGA8AO5vgmgHVbZZqQ3GkxNa8lzHNH/G66xEvPXLe03h6QptCHcOugU6SwCzmxbt2hQnvbQXZk4eGz330CmwlwVPEAKP3VwWsruEkU+LWBIohDg/0Yk6Dg==",
            "name": ".",
            "type": 46
        }
    ],
    "CD": false,
    "Question": [
        {
            "name": "example.org.",
            "type": 1
        }
    ],
    "RA": true,
    "RD": true,
    "Status": 0,
    "TC": false,
    "edns_client_subnet": "103.123.164.0/0"
}

Configure your browser

Well in Firefox this is pretty easy.
Search your preferences / settings in Firefox for DNS.

In Google Chrome the setting will be released in version 78 I’m told, I guess I’ll keep an eye out 🙃

Conclusion

So, maybe you’re super sentive about people spying on you, maybe you like playing with new gadgets / tech. Who knows.
But, I certainly don’t trust cloudflare enough to be giving them my browser DNS queries!

So we’ve now got DNS over HTTPs to use, and we own all our own history / data 👍

Install Golang On Debian Buster

2019-09-10T00:00:00+12:00

Golang, or just simply Go is an open source programming language that makes it easy to build simple, reliable, and efficient software.
This is way easier than you think !!

Step 1 - download

Firstly, let make sure our system is up-to-date
sudo apt update && sudo apt upgrade

No let’s download the go package:
Go to Go’s download page and grab a link for the latest version.
The version when I wrote this wee blog was go1.13.
Got the link? sweet! let’s download it:
wget https://dl.google.com/go/go1.13.linux-amd64.tar.gz

Now let’s untar it
tar xvfz go1.13.linux-amd64.tar.gz

And move it to an appropriate place (like /usr/local maybe?)
sudo mv go /usr/local/go

Step 2 - setup your environment

We’re going to add a few environment variables in our .bashrc, and add to our $PATH.
so let’s do that

cat << 'EOF' >> ~/.bashrc
export GOROOT=/usr/local/go
export GOPATH=$HOME/goProjects
export PATH=$GOPATH/bin:$GOROOT/bin:$PATH

EOF

and then create our goProjects directory:
mkdir ~/goProjects

and finally, let’s reload our environment:
source ~/.bashrc

What we’ve done is:

Told our bash environment where to find go
Told our bash environment where our working directory is (~/goProjects)
added two directories to our $PATH

Step 3 - verify your install

Here comes the easy bit, let’s check our installed go version:
go version

go version go1.13 linux/amd64

and let’s check our go environment
go env

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/paul/.cache/go-build"
GOENV="/home/paul/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/paul/goProjects"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build934234536=/tmp/go-build -gno-record-gcc-switches"

Conclusion

Simple eh? 👍

no get “go-ing” 😂

Linux Streaming Webcams

2019-03-11T00:00:00+13:00

In this little snippet / howto I’m going to show how I got a regular Logitech webcam streaming over the internet.

So let’s start with the problem:
I have a fishtank, I wanted to create a “fish cam” :)

Step 1 - Hardware

So I have a regular Logitech HD Webcam C910, and a standard Debian stretch sitting on a PC next to my fishtank.

Pretty straight forward so far, I do have contrib and non-free repos enabled

deb http://deb.debian.org/debian/ stretch main non-free contrib
deb-src http://deb.debian.org/debian/ stretch main non-free contrib

deb http://security.debian.org/debian-security stretch/updates main contrib non-free
deb-src http://security.debian.org/debian-security stretch/updates main contrib non-free

Step 2 - The streaming software

Motion - link

Here’s a wee quote from the website

Motion is a highly configurable program that monitors video signals from many types of cameras. Set it up to monitor your security cameras, watch birds, check in on your pet, create timelapse videos and more.

Wow, that’s what I want to do - check in on my pets :)

So let’s install it:

aptitude install motion

TADA !!!!

No let’s configure it.

I don’t want to keep images, I just want to stream, so the steps I need to do are:

Create a conf directory in my home directory
Create a config file in the created config directory

So let’s create a directory by doing
mkdir ~/.motion

No let’s create a config file by doing
vi ~/.motion/motion.conf

Note: probably don’t do this as user root

Let’s look at my config

stream_port 9091
stream_localhost off
webcontrol_localhost off
webcontrol_port 9092
minimum_motion_frames 1
output_pictures off
quality 100
stream_quality 100

Basically, the needed bits are stream_port 9091, stream_localhost off, and output_pictures off.
This sets the tcp port you want to use, disables “localhost” only, and turns off saving images.

Run the server

Here’s the fun bit.
motion

That’s it, you have a streaming server up and running. You can view it by open a browser and going to http://your.ip.address:9091/

Wrapping up

So to get this all up and running (and public if you want that) you have to open a tcp port on your firewall.
I use Mikrotik, so here is what I do:

/ip firewall nat
chain=dstnat action=dst-nat to-addresses=10.0.0.2 to-ports=9091 protocol=tcp in-interface=ether1 dst-port=9091 log=no log-prefix=""

and of course you want to see it running, so here’s a link:
Fishcam (webcam streaming)
A wee note: I built a small php proxy script that looks like this:

<?php
set_time_limit(0);
$fp = fsockopen ("my.public.ip", 9091, $errno, $errstr, 30);
if (!$fp) {
    echo "$errstr ($errno)<br>\n";
} else {
    fputs ($fp, "GET / HTTP/1.0\r\n\r\n");
    while ($str = trim(fgets($fp, 4096)))
       header($str);
    fpassthru($fp);
    fclose($fp);
}
?>

Web Development And Validation

2019-02-25T00:00:00+13:00

So I do a bit of web development, and of course have developed all 3 my own sites numerous times. I’m currently using Jekyll for my own sites (TerminalAddict.com, Loudas.com, and Paulwillard.nz), you can read about it here: Jekyll Getting Started.
I want to talk about validation.

Not all web browser are equal

This is straight from the w3 website:

While contemporary Web browsers do an increasingly good job of parsing even the worst HTML “tag soup”, some errors are not always caught gracefully. Very often, different software on different platforms will not handle errors in a similar fashion, making it extremely difficult to apply style or layout consistently.
Using standard, interoperable markup and stylesheets, on the other hand, offers a much greater chance of having one’s page handled consistently across platforms and user-agents. Indeed, most developers creating rich Web applications know that reliable scripting needs the document to be parsed by User-Agents without any unexpected error, and will make sure that their markup and CSS is validated before creating a rich interactive layer.
When surveyed, a large majority of Web professionals will state that validation errors is the first thing they will check whenever they run into a Web styling or scripting bug.

Basically, all browsers suck somehow. None of them do the right things, and they are always arguing about who is doing it right, and who is doing it wrong; boring !!

What I do

I try to write valid code ! It’s really simple.
Simple things like closing all your tags, creating valid headers, using & instead of &.

I use a few tools to check my work:

HTML Validator Chrome extension: Link here

This little tool installs as part of the Chrome developer tools, and is a simple, quick test, of whether you’ve produced valid html.
This tool uses HTML-Tidy in the background. ¹

The w3c validator: Link here

This is a fantastic tool, but interestingly, returns different results to the previous mentioned tool; it must use a different validation tool, or whatever.
Absolutely use this during development, and before signing off as complete, on any web project. ²

The Structure Data testing tool: Link here

Here’s a quote from Google:

Google Search works hard to understand the content of a page. You can help us by providing explicit clues about the meaning of a page to Google by including structured data on the page.

So Structured Data lets you define and describe your website data. I use this extensively at Loudas.com, as the data is “bloggy”, and it’s used fairly heavily here too.

Chrome “Lighthouse test”

In the Chrome developer tools there is a tab titled “Audits”. This uses a chrome tool called lighthouse.
This is awesome! Not only does it test validity of your website, but also suggested ways to make your website faster.

Challenge yourself to see if you can get 100% across all tests. But here’s a warning, you’re going to learn about ³ServiceWorkers if you don’t already know about them.

The Benefits

Well this is also simple; things tend to work better when they’re using a standard.
You’ll likely come across less browser based bugs. But as I mentioned earlier, all browsers suck, so the key word here is likely :)
Plus, you can gloat on forums about how awesome your website is, and how somebody elses is rubbish.

Actually, don’t do that last one, instead just send them this link, and say something like “bro, want some help?”.

Conclusion

Do better work that meets standards :) there are heaps of tools to help you out along the way.

Got some cool tools you use? Drop me a comment and let me know.

The HTML Validator extension has started causing an error in the console: Unchecked runtime.lastError: The message port closed before a response was received. ↩
Some of my “paid” work is terrible ! The customers don’t seem to value validation, and therefore won’t pay for my time to work through errors in their web apps (read: Wordpress installs). ↩
I use Google Chrome - Workbox as my ServiceWorker, you can read through my source to see how I use it. ↩

Comments On Blogs Vs Social Media

2018-11-06T00:00:00+13:00

So I’ve run a blog for years, since 2006. Sometimes I blog about personal adventures, sometimes I share coding and sys admin things, sometimes I just talk rubbish. So this thing has been on my mind for a while Comments

It’s so easy to comment on Facebook, Instagram, <insert social media site here>. Developers ( like me ) try really really hard to make it easy for people to comment on personal blogs too.

Look here for an example, it’s simple, there is no sign up, no “confirm your email address”, simple! Just fill out a form and click submit.

¹There’s no magic in the background either. As of writing this (2018) there are no cookies used on this site, no data collection, no database in the background where I send info to advertisers, nothing. and yet, it’s so difficult to get people to comment on our own websites?

The media landscape kinda forces us, as website owners, to share our pages on facebook, twitter, instagram, whatever … and they (the social media websites) really have captured the “comments” market.

For some reason people just don’t like commenting on a private website, but they have no problem commenting on social media sites?!?!?

The problem

So, the problem from a website owners point of view is - success breeds success. and showing that you’re successful is important to give your website credibility. If people see that others have commented, then they are more comfortable commenting.

But if that comment is on another social media site they don’t see it on the private website. :( sad face

Also there is a Google search problem.

Google is by far the largest search engine in the world. Most people find my websites through Google. BUT if your comment is on another social media site, Google doesn’t see it, that means Google thinks “Gee this site is inactive, I’ll stop recommending it to people”. :( bigger sad face

The solution

Well, I’m not sure. People need to trust a website, trust that it’s OK, that the website owner is not going to sell their details to some advertiser, or use their email address to spam.

Then, the whole world needs to shift their ideas of how to interact with other people online.

Facebook is dying, like actually .. fewer and fewer people use it. They’re sick of the advertising, the “share this if you agree” bollocks, the whole thing has just turned into a mess. Photo sharing like instagram has become popular (instagram is owned by facebook, so they’re probably selling your details like they always have) but it is no good for text blogging; Should we go back to the old fashioned days of visiting people. :)

Call in for a cuppa

Back in the bad old days, you used to call in to see your friends, share a cup of coffee, catch up, then do it all again in two weeks.

Do we go back to that idea? “I’ll just call into TerminalAddict.com and have a cuppa” :)
Sounds ridiculous, but maybe that’s just what’s needed ? Who knows?

I’d love to hear your thoughts .. leave a comment :D

I do use google analytics to track visits to my website .. but that’s it. ↩