Devops on traviscj/blog

big-ish personal storage

Wed, 25 Apr 2018 00:00:00 +0000

I was curious what it would cost to have something like 5-30TB of storage space on hand. The obvious choices are between buying some hard drives and paying for an S3 bucket or something.

The amazon costs are pretty straightforward: you pick a region, they tell you the cost. Starting with their pricing table:

aws = [
    {"level": "standard", "c_millidollars": 23},
    {"level": "infrequent", "c_millidollars": 12.5},
    {"level": "one_zone", "c_millidollars": 10},
    {"level": "glacier", "c_millidollars": 4}
]

we can say

switching over to https

Fri, 15 Apr 2016 00:00:00 +0000

One of the things I’ve been meaning to do forever is switch things over to https. By “things”, I mean the set of websites I run for some family and friends. I tried it out with my personal website first, then flipped over the rest.

implementation notes

I used the letsencrypt start guide to generate the certificates.
Modified the nginx config to: a. serve ssl/https traffic on port 443 for the given domain with the proper https certificates/etc. b. forward non-ssl/http traffic on port 80 to port 443 for the given domain

verification

It turns out that the nginx configuration files are a little bit error prone. This probably means that I am doing something wrong, like not using some configuration management tool like puppet or ansible or whatever. But for something as small scale as my site, it doesn’t really meet the cost-benefit threshold for learning a new tool/language. I also even considered spinning up a simple one-off configuration generator that I’d need to figure out how to override and extend as needed.

SSL Cert Reissue

Wed, 09 Apr 2014 00:00:00 +0000

Like many others, I have been hit by the heartbleed bug, which kinda sucks. I don’t use SSL for anything very critical, but I do use it at [tcj.io tcj.io], my “projects” website. My host, Linode, has done a great job of providing tutorials on how to deal with the situation. The obvious first step (a couple of days ago) was to upgrade openssl itself:

apt-get update
apt-get upgrade

But this only prevents the server from leaking keys going forward. Since the vulnerability was in the wild for quite some time, I thought it prudent to reissue the certificates as well. Now that I had a bit more time, I went ahead and did a reissue to make sure that nothing going forward gets leaked. This is (as usual) a bit annoying, because of the verification procedure at [Gandi gandi.net]. Otherwise, they’re pretty solid though, so I guess I’ll give them a pass on this one. And they did allow a reissue without revoking, so that’s a good step!

website basics

Fri, 15 Feb 2013 00:00:00 +0000

Several of my classmates in graduate school are considering making websites around now, and have asked me to explain it. I figured I’d explain the stack as I have it set up, and where you could make tradeoffs. For now, I’m not going to include too many detail here; I just want to outline the possibilities and lingo so that researching options is a bit easier for the uninitiated.

First off, many people are perfectly happy to use the school’s email and department web space. If you only want to have a list of relevant papers and soforth, then this is a fine approach. See the very bottom of this post for a couple potential ideas for organizing it.

tjtestharness - a language-agnostic DVCS unit-test/continuous integration tool

Mon, 26 Mar 2012 00:00:00 +0000

I’ve been wanting to have a way to visualize which unit tests(or sets of them) passed for a given commit, if for no other reason than the sense of accomplishment from watching boxes turn yellow, then green, as they pass tests. The trouble is, I write code in a lot of different languages for a lot of different projects. I also don’t want to bother with running unit tests individually–I want them to run as I perform commits.

GenHosts

Tue, 24 Feb 2009 00:00:00 +0000

Winter quarter 08, the hosts.allow files used by tcpwrappers was getting unwieldy. For one thing, we had certain groups of hosts that were all allowed to connect to eachother anywhere, some groups that were only allowed one port on one computer, some that were allowed certain parts of machines. It was basically a mess to try and keep up with it. So I wrote a set of scripts to let us update one place and have it synchronize everywhere else, all in very simple XML files.

Johnson vs Acronis, Pt 2

Mon, 04 Aug 2008 00:00:00 +0000

Some more ways Acronis fails, just for the record:

Completely useless log files. Error messages are crucial in applications. Acronis says ‘Error creating file’. This could mean:
I couldn’t resolve the FTP server’s hostname
I could resolve the hostname, but could not connect to the FTP port
I could resolve the hostname and connect, but I could not authenticate.
I could resolve, connect, authenticate, but I could not create a new file.
Any number of other things. I could not create a temporary file(Why? No permissions in Acronis’s temporary folder? No hard drive space? It already exists? The folder that should be the temporary folder doesn’t exist?)

Another error we’ve gotten a few times is ‘Failed to process pair script (some hash).’ While it might be useful to someone, it’s not really any help. A lot of the same stuff applies: Is the file corrupt? What’s wrong with it? Improper syntax? The configuration files are XML, they should be editable as such. Point being: Acronis, get your stuff together and write some real, honest error checking code. It’s boring code to write, but it’s boilerplate stuff, and basic. This is what you hire interns for. Please do so. 2. The GUI stinks. Everything is wizard based. This in itself isn’t so much of a problem, but it becomes one with their absolute awful support of stuff like, oh, say, remembering passwords. To see how this is an issue: Say I realize that I want maximum encryption on my backup archives. So, I load up the tasks list, click the one I want to change, click the ‘changed settings’ button. It asks me what I want to backup(2-3 screens), then where I want to back it up to. I tell it the FTP site, and it asks me for my login for that site, then allows me to navigate(keep in mind this is already stored in the configuration file I’m trying to edit, in some encrypted form(hopefully, though I would not bet 2bits on it)). Then, it asks me if I want to sign in to the FTP anonymously or use a username and password. Upon selecting non-anonymously, it asks me for the same username and password. Every single time you try to change any part of the configuration. 3. The command line stinks. There are just a couple of commands, including trueimagecmd and trueimagemnt, but both of them are completely inconsistant on file argument parsing: trueimagemnt uses the more standard UNIX conventions of [-(shortopt)|–(longopt)] (arg) while trueimagecmd uses the completely zany –(longopt):(arg). The command trueimagecmd allows a –verify command, but doesn’t include it in the runtime help or the manual page. None of this seems like too big of a deal, but Acronis is basically the only program that we actual require any GUI at all for on 2 of our 4 main servers. It’d be very nice to have backups and not need a GUI.

Johnson vs Acronis

Sat, 02 Aug 2008 00:00:00 +0000

The story goes something like this: Originally, we used external hard drives plugged in via USB to our servers to back up our servers. This worked well, except that we had two external drives and 7 servers. About this time I started working this job and immediately set up a MediaWiki website for storing documentation and a Mantis Bug Tracker website for storing information about ongoing projects–I think I’ll try to write a post about that at some point as well. Anyways, these two websites originally ran on my personal webserver in Prosser, but we decided to run them on a company-owned server instead, and that this new machine could act as both a backup server and a webserver for those two websites.

NSA's RHEL5 Guide

Mon, 07 Jul 2008 00:00:00 +0000

One of my tasks at work is to write up a security checklist of sorts, and from one of Ryan’s notes, I happened across the NSA’s Red Hat Enterprise Linux 5 guide. It’s pretty cool and covers basically everything. Have a look

Acronis TrueImage

Sat, 30 Jun 2007 14:00:00 +0000

TrueImage is… well… Quite the piece of software.

What I need it to do: Tell the check_backupdisk script that it’s actually running the backup (IE–don’t bitch on nagios) and handle all the mounting/umounting/fscking that it currently does. I think I’ll probably end up putting a wrapper script around it…. but we’ll see.

MRTG

Sat, 30 Jun 2007 13:00:00 +0000

So, another day, another (minor) problem. I had connected to a server with KDE instead of our usual GNOME interface. Which probably would have been fine except that KDE had a screensaver that sucked up the CPU. Which also would have been fine, if it hadn’t been a server that everyone else used. Oops. RRDtool and MRTG would have shown a CPU spike which I could have investigated.

Cacti

Sat, 30 Jun 2007 12:00:00 +0000

So, rrdtool is pretty sweet, but it’s somewhat of a pain to set up. So… we use Cacti. It’s amazing. Supposedly it’ll integrate with Nagios as well, and then all of this data will be at our fingertips. For now, I just set it up on my server that’s probably going to go down shortly. In any case, I feel like this project is somewhat coming together. Nagios is great for what’s going on right now, cacti needs a way to know what’s going on right now–A match made in heaven, or something.

RRDTool

Fri, 29 Jun 2007 14:00:00 +0000

I have apparently been missing out. There’s a really neat little unix tool called ‘rrdtool’ that uses a fixed-size database to look at how something changes over time. I think I’ll use it to monitor server load… and possibly network traffic.