I just finished some basic load testing/benchmarks of DNS servers for Phyber. The results are suprising and available on Phyber’s Blog here:
August 19, 2010
August 18, 2010
HOWTO Subnet IPv6 for Network Links
I’ve previously posted some information on IPv6 subnetting on my personal blog http://www.clarksys.com/blog/2009/03/12/howto-subnet-ipv6/. With Phyber’s recent expansion into our One Wilshire Annex space the opportunity presented itself to update and correct some of the initial IPv6 allocation and configuration that was done.
First some background… an IPv6 address is comprised of eight (8) blocks of four hexadecimal digits separated with a colon “:”. Each digit can be either a number [0-9] or a letter [a-f] for a total of 16 possible combinations per digit. 16^4 (yes this should be written 2^16) gives 65,536 possible combinations per block. The double colon “::” abbreviation is commonly used in place of all zeros. This double colon can only be used once while specifying and address.
The default IPv6 allocation from a RIR for an ISP is a /32. For reference Phyber’s subnet is 2607:f238::/32 and will be used in this post. Sipcalc (a very cool utility) spits out this information on the Phyber subnet:
MaxMini:~ mclark$ sipcalc 2607:f238:0:0::/32
-[ipv6 : 2607:f238::/32] – 0
[IPV6 INFO]
Expanded Address - 2607:f238:0000:0000:0000:0000:0000:0000
Compressed address - 2607:f238::
Subnet prefix (masked) - 2607:f238:0:0:0:0:0:0/32
Address ID (masked) - 0:0:0:0:0:0:0:0/32
Prefix address - ffff:ffff:0:0:0:0:0:0
Prefix length - 32
Address type - Aggregatable Global Unicast Addresses
Network range - 2607:f238:0000:0000:0000:0000:0000:0000 -
2607:f238:ffff:ffff:ffff:ffff:ffff:ffff
-
Notice the Expanded and Compressed addresses above.
The next thing to point out is the major IPv6 subnet boundaries. For example:
2607:f238:0000:0000:0000:0000:0000:0000
| | | | | | |
| | | | | | - /112 Subnet
| | | | | |
| | | | | - /96 Subnet
| | | | |
| | | | - /80 Subnet
| | | |
| | | - /64 Subnet
| | |
| | - /48 Subnet
| |
| - /32 Subnet
|
- /16 Subnet
I’ve included /80, /96 and /112 for reference but I am going to tell you to completely ignore them and I will explain why.
In my initial IPv6 subnetting I was faced with two initial issues a) what do I do about our router loopback addresses, and b) what do we do about the point to point links. The first was easy, in IPv6 a single host is represented as a /32 this maps directly to an IPv6 /128. The second not so much. With IPv4 we have specific space constraints and as such have been conditioned and use /30 subnets for links (two hosts, with a network and broadcast address = four IPs). A /30 would be a /127 in IPv6 parlance, however a quick Internet search will point out all of the potential evils of using /127s and tell you to use /126s instead. And at the same time tell you to always, and I stress ALWAYS use /64s when assigning a network.
Now a /64 is a huge amount of IP space. It’s 2^64 or 18,446,744,073,709,551,616 IPs to be exact. That’s a crazy huge number, in fact I had to look up how to pronounce it. 18 sextillion, 446 quintillion, 744 quadrillion, 73 trillion… IPs. The idea of using a network with 18 sextillion IPs in it for a router to router link seemed a little crazy to me. And then I found this presentation by Matsuzaki ‘maz’ Yoshinobu:
IPv6 address architecture on point-to-point link
The presentation is a quick read and spells out the technical difference between a /127 and a /126 and reinforces the concept use of a /64 for all networks.
This was the eureka! moment for me, and the following points became crystal clear:
- An ISP is allocated a /32
- The ISP is supposed to break this into /48s for customers (there are 65,536 /48s in a /32)
- The /48s should be subnetted into /64s (- )there are 65,536 /64s in a /48)
- Always us /64 subnet boundaries for everything
Even “wasting” IPv6 addresses, how many networks have more than 65,536 network links? If you do, you can always allocate another of your 2^16 /48s available.
So for Phyber this resulted in the following network plan:
2607:f238:0000::/48 allocated for internal use
2607:f238:0000:0000::/64 allocated for loopback addresses
2607:f238:0000:0001::/64 allocated for internal server segment
2607:f238:0000:0002::/64 link #1
2607:f238:0000:0003::/64 link #2
2607:f238:0000:0004::/64 link #3
2607:f238:0000:0005::/64 link #4
2607:f238:0001::/48 first customer
2607:f238:0002::/48 second customer
2607:f238:0003::/48 third customer
and so on and so on.
If you refer back to the presentation I mentioned earlier there’s notes about the potential dangers of /64s on network links and why people intuitively want to subnet to a /127 or a /126. We ended up splitting the difference and actually subnetting the /64 for the network link to a /126.
IPv6 is a very large pool of IP space – to paraphrase my favorite quote so far “IPv6 has 340 undecillion unique addresses (that’s a 39-digit number). If IPv4 is a golf ball IPv6 is the sun.” Trust me, don’t try to over think this. Just follow what all the RFCs say and use /64s for your network links.
If you want to read more I found the following links to be very helpful in understanding how to properly subnet IPv6:
September 3, 2009
Pass a PCI Compliance Scan in 5 Steps
PCI (Payment Card Industry) Compliance/Standards were originally created to standardize on a set of rules and guidelines to help protect credit card holders from theft – specifically theft from a website being hacked and their information compromised. At what point PCI Compliance was bastardized I do not know – but today it’s really a joke.
In order to maintain a merchant account and process credit cards, most vendors will require a passing score on a compliance scan. There’s nothing quite like a little regulation to create a new industry – today a search on Google for “PCI Compliance Scan” returns almost 3/4 mil results. Don’t get me wrong, I am a firm believer in the standards put forth by PCI, but the requirement to pass an automated scan is nonsense. Almost all companies offering PCI compliance are using the Nessus security scanner which connects to your machine, runs through an automated scan and spits out a report with a score.
What kind of things should you expect to find on the results? Gems like:
Synopsis : It is possible to determine the exact time set on the remote host. Description : The remote host answers to an ICMP timestamp request. This allows an attacker to know the date which is set on your machine. This may help him to defeat all your time based authentication protocols. Solution: filter out the ICMP timestamp requests (13), and the outgoing ICMP timestamp replies (14). Risk Factor: Low / CVSS Base Score : 0 (AV:R/AC:L/Au:NR/C:N/A:N/I:N/B:N) CVE : CVE-1999-0524
And:
Your computer appears to be running http software that allows others to view its web pages. If you don’t intend this computer to allow others to view its web pages then turn this service off. There are many potential security vulnerabilities in http software.
My general advice to computer security applies here. If you don’t need/use a service/application then turn it off, and keep your computer up to date with the latest security patches from the vendor. Fedora and CentOS users can just run “yum update”.
Anyways, back to the scan…
1. Run the initial scan against your site – you will probably fail. Open your web server logs and find the IP address of the machine that scanned you. Save this IP for later.
2. If you are running PHP you will have lot’s of security warnings, add this line to your /etc/php.ini (or wherever it may be):
expose_php = Off
3. Disable Trace, Track, Server Signatures and Server Tokens in Apache (/etc/httpd/conf/httpd.conf):
TraceEnable off
ServerSignature Off
ServerTokens ProductOnly
4. Now the fun part, using the IP address you found in step one block the entire Class C (/24) IP Netblock range from connecting to your server on ports other than 80 and 443 (http and https). In /etc/sysconfig/iptables immediately after this line:
-A RH-Firewall-1-INPUT -i lo -j ACCEPT
Add these entries:
# Filters for PCI Scan
-A RH-Firewall-1-INPUT -s x.x.x.0/24 -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT
-A RH-Firewall-1-INPUT -s x.x.x.0/24 -m state --state NEW -m tcp -p tcp --dport 443 -j ACCEPT
-A RH-Firewall-1-INPUT -s x.x.x.0/24 -j REJECT --reject-with icmp-host-prohibited
What this does is allow the scanning system to connect to the web server via http and https only – everything else will be denied.
5. Reload IP Tables (service iptables restart) and Apache (service httpd restart) – and rescan your site.
September 2, 2009
Using IRR with Level3
Level3 has a very specific/finicky RPSL based router configuration tool. So specific in fact if you have your route objects in RADB and include customer objects from ALTDB the ALTDB objects will be filtered. If you are having this problem – here’s the secret, Level3 needs to add additional sources to their search path, enable recurseok and warnonly. Example below:
import policy:-le=32 RADB::AS-PHYBER -searchpath=RADB;ALTDB -recurseok -warnonly
September 1, 2009
Set the time zone on a per user basis
Continuing on my general rant that all computer systems should use UTC/GMT for their system clocks the question is often posed by users that want to see their own local time when they log into a remote computer.
Procedure – set the time zone on a per user basis
Open user ~/.bashrc or ~/.bash_profile file using vi text editor and set up TZ environment variable. Append or SET TZ as follows:
export TZ=”/usr/share/zoneinfo/{TIMEZONE-DIRECTORY}/{TIMEZONE_FILE}>”
If your username is foo and you would like to set TZ to Asia/Calcutta (INDIA IST) type command:
# vi /home/foo/.bashrc
Append following:
export TZ="/usr/share/zoneinfo/Asia/Calcutta"
Save and close the file.
(via http://www.cyberciti.biz/faq/howto-linux-set-time-zone-per-user-basis/)
June 19, 2009
One Less Email Account?
Like many on the interweb, over the years I have collected one or two email accounts. Most of them come free with registration to a service or site that I use for some piece of functionality (think schools, flickr, instant messenger, storefronts, etc…) – others are just part of registering a domain and publishing a site.
Managing this mess of accounts has always proved to be a challenge typically solved (I use solved loosely here) with the use of Sendmail/Qmail/Postfix virtual hosting + Thunderbird and IMAP/POP3 aggregating into a single set of folders on my desktop. In the past couple of years I have found myself using and preferring Google Mail for my personal accounts, both with their @gmail brand for mailing lists and newsgroups and also with the Google Apps for my custom domain hosting. While this has given me simple care free hosting it hasn’t solved the underlying issue of just too many accounts to manage and filter through and the time drain involved.
So today in an attempt at efficiency I have reshaped my inbound email to simplified email forwarding modeled after pobox.com (I thought about using their service but it literally costs me $0 to host on one of our many existing email servers). For the time being I will use my @gmail account as my primary personal inbox while maintaining my @clarksys account for permanency.
If you’d like to send me an email (click here to see my address) – you could probably guess it anyway.
May 19, 2009
There’s Never a Dull Moment
There are no words… I was just forwarded a call from our NOC after the caller demanded to speak with the manager. I am still not clear what the issue is but the caller “Carlo” was demanding to speak with Juan. I repeated the same thing that Heather had said – that there was no person named Juan that worked for the company. Carlo goes on about how I’m a lier and that Juan is a security guard in my building. Okay so at this point I’m thinking Carlo must be talking about one of our datacenters which are all staffed by security. Before I can respond Carlo starts in that I’m a lier and he’s going to send a couple guys down to sort this out.
I still have no idea:
- What building he’s talking about
- Where he got our number
- What his original issue was
- What exactly he thinks he’s going to accomplish by sending some guys out to deal with building security. I’m not positive but I would guess that Securitas/Pinkerton has 911 on speed dial.
Unfortunately my curiosity is peaked and I doubt I’ll ever find out what exactly is going on.
May 4, 2009
HOWTO Install OS X on a Dell Mini 9
I just finished installing OS X on my Dell Mini 9 pic (as an aside – probably one of the best form factors I have owned for a traveling laptop). The instructions here:
How-To: Install Mac OS X – DellEFI Method
are pretty good. I followed Section 2 Part A. One thing to note, when booting the system via the “small” USB drive you will have to press “esc” and select the larger USB drive to boot from. The syslinux/isolinux on the small USB drive will continue to boot in a loop until you break the cycle. Other than that everything worked like a charm right from the start (camera, audio, wifi, etc…).
I still need to install and tweet some additional settings and my 16 GB USB drive will always be close by (6 GB installation). Additional links and information are below – enjoy.
May 1, 2009
Patch for IRR Power Tools (irrpt)
Small patch to the IRR Power Tools (irrpt) package to allow setting the prefix length via the irrpt_pfxgen script on the command line:
--- irrpt-1.27/bin/irrpt_pfxgen 2007-06-08 21:16:47.000000000 +0000
+++ irrpt/bin/irrpt_pfxgen 2009-05-01 19:36:30.000000000 +0000
@@ -50,7 +50,7 @@
case "-l":
case "--pfxlength":
- $o_pfxlength = $_SERVER['argv'][++$offset];
+ $o_pfxlen = $_SERVER['argv'][++$offset];
break;
case "-f":
April 27, 2009
The Real Money is in the Cross Connects
About a year ago I heard a stat that One Wilshire had 40 Gbit/s of utilization on their Fast Ethernet cross connects. Honestly this number floored me for a few reasons:
- an average circuit will run at 30% utilization, this means that CRG had approximately 1,300 Fast Ethernet circuits run at the time. Or in revenue numbers, been $65k/month in fast ethernet cross connect fees.
- Fiber cross connects are the overwhelming majority of installed circuits at any carrier neutral site. I couldn’t say what the exact ratio is but based on my personal experience Fiber to Ethernet is something on the order of 20 to 1. This would mean One Wilshire was billing $2.6/month in fiber cross connects.
- One Wilshire is a relatively inexpensive location for cross connects, other facilities are way more expensive and could be billing way more.
While the largest revenue items for a given facility will most certainly be floor space and power, those also carry the highest cost of goods sold. With cross connects the facility bears a one time charge for the materials and labor (which is usually billed to the customer as an installation fee) and after that there is almost zero expense ongoing to service the cross connect. The margin is nearly 100%.
Like I said, the real money is in the cross connects.
