February 21, 2019

smart arguments

It’s easy to love the terseness one can get out of a command-line tool with a “pass by order” convention. For example, maybe I run something like

deploy myapp sea2 production

to deploy an application called myapp to the second production datacenter near Seattle. This works great, except now I need to remember the order of arguments is: deploy [app] [datacenter] [environment].

The typical solve for this problem is introducing named arguments, so you’d end up with

January 8, 2019

feed sequences

In the mysql feeds post, I mentioned that the publisher could do

SELECT MAX(feed_sync_id)+1 
FROM kv

to find the next feed_sync_id during the publishing process, but this is actually a really bad idea. (And I knew it at the time, so forgive me for selling lies…)

Republishing

Before we jump into the problematic scenario, I’d like to motivate it with a tiny bit of background.

The republish operation is extremely useful when consumers need to receive updates. It is also extremely simple! A query like

January 7, 2019

pattern: extract mutating variables to State class

I’ve come to really like the pattern of extracting local or instance variables into their own State class, and writing an algorithm in terms of that State.

A really trivial (if contrived) example of this would be a “sum cubes” algorithm; something like this:

public class CubeSummer {
  private final PrintStream printStream;
  private long i;
  private long sum;
  
  public CubeSummer() {
    this.printStream = System.out;
    reset();
  }
  
  public long sumCubes(long limit) {
    for (; i<limit; i++) {
      sum += i * i * i;
      display();
    }
    return sum;
  }
  
  public void reset() {
    i = 0;
    sum = 0;
  }
  
  public void display() {
    printStream.print("i = " + i + ";");
    printStream.println(" sum = " + sum);
  }
}

What’s happened here is we’ve essentially promoted some variables that should arguably be local variables into instance variables, to avoid needing to pass them around explicitly. But in the process, we’ve also mixed long-lived state with short-lived state, broken thread safety, and forced clients of this class into a particular calling convention (either instantiate an instance per sumCubes call or carefully respect thread safety and call #reset().)

January 6, 2019

unreasonable wish -- an sql interface to websites

I was searching Amazon Prime for some Earl Grey tea for my wife. I got these results

earl grey tea

This was just a basic search for earl grey tea in the Prime Pantry store.

I would love to

SELECT *
FROM products p
WHERE p.prime_pantry = 1
  AND p.keywords = 'earl grey tea'
  AND p.packaging = 'bulk'

I get that this’ll never happen, but man… it’d be nice.

January 5, 2019

more light, better pictures

We were trying to take some Christmas pictures but feeling a bit unsatisfied with the way the pictures were coming out:

without_lamp_light_s

I wanted some flash or something, but the batteries on all the “better” cameras were already dead, so I went looking for an alternative. We tried to get the overhead light turned on, but couldn’t find the switch for that.

The next-closest source of light was a sheet music lamp on my mom’s piano! Borrowing that for the cause, I was able to get this one instead:

January 4, 2019

watching progress in mysql

Pretty frequently at work, I end up polling a database with some command like

SELECT MAX(id) FROM my_tbl;
SELECT MAX(id) FROM my_tbl;
SELECT MAX(id) FROM my_tbl;
SELECT MAX(id) FROM my_tbl;
-- .... ad nauseam ....

I’ve eventually noticed a few patterns I use pretty consistently:

estimate ETA by including `NOW()`/`UNIX_TIMESTAMP()`

Generally, the point of hovering over the table is to get an estimate of when it will finish/catch up/whatever. For that, you generally want to include a timestamp in the query output, so when you come back a while later, you can know exactly how much time has elapsed. I might transform the above query into

January 3, 2019

personal finance resources

I wanted to say a tiny bit about personal finance stuff, as I feel like I have gotten a fairly good feel for it.

Some great resources to start getting up to speed include:

r/personalfinance, especially
- the common topics
- the wiki

index investments

Ideally you want to own a lot of different companies, so you’ve diversified your asset ownership. One of the easiest ways to do this is through index investing, where the index “tracks” a broad swath of the market, like “All S&P 500 Companies” or “All stock market companies”.

January 2, 2019

espps are free money

ESPPs give employees an opportunity to buy the company stock at a discount. In both of the examples I’m aware of, the companies give a 15% discount on the LESSER of the price on the grant date and the price on the purchase date. The purchase dates are every six months, while the grants I’ve seen are either 12 or 24 months.

We can analyze this mathematically by breaking it into three cases. For concreteness, let’s look at ADBE for a grant date of 2019-01-02. The stock is trading at $224.27/share currently:

January 1, 2019

tradeoffs

Life is (just) a series of tradeoffs.

DawnAnn “Mom” Johnson

I heard this so many times growing up that I think I actually stopped hearing it. But recently, I realized that it’s never been more pertinent than to my current day-to-day life as a software engineer.

The fundamental tradeoff we make on the Risk Systems team is trading off between

how much fraud we catch (and symmetrically, miss!)
how many false positives we incur to catch that much fraud

Those false positives running rampant can inhibit growth (in terms of interested customers) of the product we’re trying to protect, but letting too much fraud through can make a product too expensive (and even liable to be shut down!)

October 29, 2018

sharded feeds

Suppose that:

our humble KV feed sees a lot of traffic.
someone needs to consume our KV feed with multiple threads.

data

The first step is to introduce a notion of “shards” into our data model:

ALTER TABLE `kv`
  ADD COLUMN `shard` INT(11) DEFAULT '0',
  ADD INDEX `k_fsi_s` (`feed_sync_id`, `shard`);

publishing

We don’t need to alter the publishing until the publishing itself is too slow to work with a single thread, but this introduces a lot of complications, so let’s just hold off for now.