October 17, 2018

jquery selectors

This is silly, but I always forget this: I also learned you can specify things like // @require https://code.jquery.com/jquery-2.1.4.min.js in a Tampermonkey script, even when a given site doesn’t already have jquery.

October 7, 2018

Naming is hard (so don't)

A lot of times I just want to record something. This should be one of those things computers are good at. Turns out this is a bit harder than it seems: many editors make you name a file to save it. One easy-sounding way to record something is: open up a text editor mash the keyboard until the thought is out of your head and into the text editor (hard part starts) what do we name that file?

October 3, 2018

Feeds as cache invalidation mechanism

One really cool use of feeds we’ve realized is that it gives a very efficient mechanism for application code to load the most recent versions of a table into memory. The basic idea is: Set it up as a usual feed published table with an appropriate index on feed_sync_id. Either alongside or within the cache, represent the latest loaded feed_sync_id. Set up a cronjob/etc that reads the latest feed_sync_id and compares it to the cache’s feed_sync_id.

July 11, 2018

robustness principle and mocks

The Robustness Principle (or Postel’s Law) states Be conservative in what you do, be liberal in what you accept from others (often reworded as “Be conservative in what you send, be liberal in what you accept”). This principle has some criticisms. I realized this has interesting implications for mocks. Suppose you have public class MyObj { private final Integer x; public MyObj(Integer x) { if (x < 0) { throw new IllegalArgumentException("negative x!

July 10, 2018

cross-dc sync with feed published KV

It’s been fun describing the feeds framework we use at Square. Today we’ll dive into a concrete problem: We’ll stick with the feed-published kv table again. We want two instances of some application code to bidirectionally synchronize the writes that happened on their instance to the other. Eventually consistent is ok. First, a bit of justification, though: I use this KV table to remember things I might have to look up without usual context, like my motorcycle’s license plate number, or that one weird python snippet I can never remember.

July 2, 2018

history preserving data models

Start with a super simple data model: CREATE TABLE kv ( id BIGINT(22) NOT NULL AUTO_INCREMENT, k VARCHAR(255) NOT NULL, v LONGBLOB NOT NULL, PRIMARY KEY (`id`), UNIQUE KEY u_k (`k`) ) Engine=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin Suppose we want to audit “changes” to this data model. Approach 1: kv_log add data model like CREATE TABLE `kv_log` ( id BIGINT(22) NOT NULL AUTO_INCREMENT, changed_at TIMESTAMP NOT NULL, k VARCHAR(255) NOT NULL, old_v LONGBLOB NOT NULL, new_v LONGBLOB NOT NULL, ) current value query: unchanged

June 29, 2018

mysql feeds

At work, we use a pattern called feeds that gets an incredible amount of work done. I’ve been wanting to describe it here for quite a while, and now seems as good of time as any. The basic premise is: You have a service A with some data that other “consuming” services B, C, and D want to find out about. Maybe the data is payments, maybe it’s support cases, maybe it’s password changes… whatever.

May 31, 2018

constraint violations in tests

I had some test code that looked something like @Test public void a() { db.insert(0, "a"); assertThat(db.query(1).getKey()).isEqualTo("a"); } @Test public void b() { db.insert(0, "a"); db.insert(1, "b"); assertThat(db.query(2).getKey()).isEqualTo("b"); } @Test public void c() { assertThat(db.query(3)).isNull(); } There’s some stuff to like about this: pretty straightforward which test is touching which records pretty clear what you expect to see in the database. There’s a bunch of coincidences here that make this work, though:

May 15, 2018

multiple thread test cases in java

Some of my day job lately has been trying to get a better handle on threads in Java. In particular, we had a really weird race condition resulting from multiple threads processing an input data stream and causing some unexpected conflicts in our data access layer. To try to replicate the problem in tests, I started with something like this: private static Runnable runWithDelay(String name, long millis) { return () -> { try { Thread.

April 25, 2018

big-ish personal storage

I was curious what it would cost to have something like 5-30TB of storage space on hand. The obvious choices are between buying some hard drives and paying for an S3 bucket or something. The amazon costs are pretty straightforward: you pick a region, they tell you the cost. Starting with their pricing table: aws = [ {"level": "standard", "c_millidollars": 23}, {"level": "infrequent", "c_millidollars": 12.5}, {"level": "one_zone", "c_millidollars": 10}, {"level": "glacier", "c_millidollars": 4} ] we can say