Max Clark Entrepreneur, Investor, Thrill Seeker

The Power of FreeBSD

As an odd consequence of being so stable. I find our FreeBSD boxes are always running EOL software. Then the struggle starts of not wanting to upgrade because the box works perfectly fine.

This one for example is being upgraded to 10.0-RELEASE via a quick stop at 9.2-RELEASE:

data01# uname -a FreeBSD data01 9.0-RELEASE FreeBSD 9.0-RELEASE #0: Tue Jan 3 07:46:30 UTC 2012 amd64 data01# uptime 5:03PM up 878 days, 19:13, 2 users, load averages: 0.74, 0.46, 0.37

This box and it's sister are being left alone:

sys01[~]% uname -a FreeBSD sys01 8.1-RELEASE FreeBSD 8.1-RELEASE #0: Mon Jul 19 02:55:53 UTC 2010 i386 sys01[~]% uptime 5:08PM up 1396 days, 18:39, 1 user, load averages: 0.00, 0.00, 0.00

What's amazing is that their uptime would probably be in the six to seven year range if it wasn't for a datacenter move.

Open Source Progress

I've been doing some spring cleaning in my hard drive and stumbled across these Apache + modssl/OpenSSL + modperl/Perl installation instructions from over a decade ago. It's really remarkable how far Open Source has come in that time.


#   extract the packages
$ gzip -d -c apache_1.3.x.tar.gz | tar xvf -
$ gzip -d -c mod_ssl-2.8.x-1.3.x.tar.gz | tar xvf -
$ gzip -d -c openssl-0.9.x.tar.gz | tar xvf -
$ gzip -d -c mod_perl-1.xx.tar.gz | tar xvf -
$ gzip -d -c mm-1.1.x.tar.gz | tar xvf -

#   configure and build the OpenSSL library: 
$ cd openssl-0.9.x
$ sh config \
$ make
$ make test
$ cd ..

# configure and build the MM Shared Memory library
$ cd mm-1.1.x
$ ./configure --disable-shared
$ make
$ cd ..

#   apply mod_ssl to Apache source tree
$ cd mod_ssl-2.8.x-1.3.x
$ ./configure \
$ cd ..

#   apply mod_perl to Apache source tree
#   and build/install the Perl-side of mod_perl
$ cd mod_perl-1.xx
$ perl Makefile.PL \
      EVERYTHING=1 \
      APACHE_SRC=../apache_1.3.x/src \
      USE_APACI=1 \
      PREP_HTTPD=1 \
$ make
$ make install
$ cd ..

#   increase the Apache hard server limit from 256 to 1024
$ vi apache_1.3.x/src/include/httpd.h

#   build/install Apache with mod_ssl and mod_perl
$ cd apache_1.3.x
$ SSL_BASE=../openssl-0.9.x \
  EAPI_MM=../mm-1.1.x \
  ./configure \
      --prefix=/path/to/apache \
      --enable-module=ssl \
      --activate-module=src/modules/perl/libperl.a \
      --enable-module=perl \
      --enable-rule=SSL_EXPERIMENTAL \
      --disable-rule=SSL_COMPAT \
      --enable-module=rewrite \
      --enable-module=so \
      --disable-module=userdir \
$ make 
$ make certificate
$ make install
$ cd ..

#   cleanup after work
$ rm -rf mod_perl-1.xx
$ rm -rf mod_ssl-2.8.x-1.3.x
$ rm -rf apache_1.3.x
$ rm -rf openssl-0.9.x
$ rm -rf mm-1.1.x

Puppet Environment Errors

This morning I found one of our Puppet installations in a state of total fail. Specifically:

May 25 16:22:47 www-app01 puppet-agent[7779]: Could not send report: Error 400 on SERVER: no 'environments' in {:rootenvironment=>#<Puppet::Node::Environment:0x7fb712b7e780 @manifest="/", @modulepath=[], @name=:"root", @watching=true, @configversion=nil>, :currentenvironment=>#<Puppet::Node::Environment:0x7fb712b7e780 @manifest="/", @modulepath=[], @name=:"root", @watching=true, @configversion=nil>} at top of [[0, nil, nil]]

A few things stood out immediately. How did a working Puppet installation go completely belly up after running without problems for so long? Apparently Puppet updated itself on May 23rd from 3.6.0 to 3.6.1 - even without any configuration in a module/class instructing it to do so (note to self, look into this later).

After banging my head against Google search for a while I noticed a new directory $configdir/environments. According to the Puppet documentation you can enable environments in one of two ways, but you still needed to make configuration changes. As an experiment I renamed the "environments" directory and Puppet started processing again.

This is just another in a long, long series of issues with Puppet that are driving me away from the platform. I get the move fast and break things approach to development - and I want you to do so - just don't completely break things on minor releases.

95th Percentile Billing

What is the 95th percentile?

The 95th percentile is a method for metering bandwidth that allows a customer to burst over their Committed Information Rate (CIR). Unlike a fixed network link, a customer is able subscribe to a CIR at a fraction (usually 10-20%) of the interface speed, but when necessary burst above and even consume the entire interface. The 95th percentile is an alternative to either fixed or GB transfered billing methods and is ideal for datacenter applications.

Every five minutes the network interface is sampled for the total amount of bytes transfered. This is averaged over 300 seconds to estimate the average transfer rate per second. These averages are collected every five minutes and stored in a database. When the month is over, the samples are arranged from highest to lowest and the top 5% of samples are removed. The next highest sample is the 95th percentile.

A 30 day month has 36 hours of free peak traffic.

30 days * 24 hours * 5% = 36 hours

The 95th percentile is not a mean or an average, its represents the value that your bandwidth is at or under 95% of the time. To avoid bursting charges you can think of this one of two ways:

  1. 95% of the time the usage should be at or below the CIR, or
  2. 5% of the time the usage can be over the CIR.

If you are looking for more information, I'd recommend you start with Wikipedia's article on Burstable Billing.

Affording Affordable Health Care

As I worked through changes to my health insurance due to the Affordable Care Act (known as Obamacare) I started wondering how this change will impact low income and part-time workers. There have been several stories quoting the reduction in benefits paid to employees due to the ACA. Even more discussing shifts from full time to sub 30 hour weeks to avoid the employer mandate requirement.

Assuming an average hourly wage of $15.00, a person working 50 hours per week in two jobs makes $37,500 per year. Single filers pay 25% ($9,375) in Federal Taxes (it's actually better to make $36k/year due to the lower tax bracket) plus 8.00% ($3,000) in California taxes. This breaks down to a take home pay of $25,125/year or $2,093.75/month. I checked - there is no premium assistance at this income level. The cheapest Bronze level plan I could find is $175/month with an estimated total annual costs for premium + out of pocket expenses of $2,494 ($207.83/month).

Think about that for a second... $2,093.75/month in take home pay with $207.83/month in mandatory health care expenses.

I clearly remember my paychecks when I was making $40k/year. I have no idea how I would ever have afforded to pay healthcare at these costs. I could barely pay my rent, car, gas, insurance and still eat.

There's a lot of good, and a lot of bad in our healthcare system. As with many things in this country, the more money you have the better off you will be. That doesn't justify a system where the poor are left out in the cold by any means. The verdict is still out on the implementation of the ACA, and until the employer mandate kicks in next year we won't really know if it's working.

One thing is certain, a lot of people are paying more for heath care than they were a year ago. Personally I went from $196/month to $276 for the same benefits.