October 17, 2018

killing all timeouts in js

this article has a nice trick for killing all javascript timeouts:

function stopAllTimeouts() {
    var id = window.setTimeout(null, 0);
    while (id--) {
        window.clearTimeout(id);
    }
}

which can be entered on the javascript console and then run with

stopAllTimeouts()

This is an effective way to prevent javascript timeouts from doing a whole variety of things, like

carousels
endless scrolls
any other kind of animations

October 17, 2018

jquery selectors

This is silly, but I always forget this:

I also learned you can specify things like

// @require https://code.jquery.com/jquery-2.1.4.min.js

in a Tampermonkey script, even when a given site doesn’t already have jquery.

October 7, 2018

Naming is hard (so don't)

A lot of times I just want to record something. This should be one of those things computers are good at. Turns out this is a bit harder than it seems: many editors make you name a file to save it.

One easy-sounding way to record something is:

open up a text editor
mash the keyboard until the thought is out of your head and into the text editor
(hard part starts) what do we name that file?
where do we put that file?
do we remember to save that file, given the difficulty of (3) & (4)?

I think picking names too early is at best a minor annoyance, and at worst a pretty major distraction.

October 3, 2018

Feeds as cache invalidation mechanism

One really cool use of feeds we’ve realized is that it gives a very efficient mechanism for application code to load the most recent versions of a table into memory. The basic idea is:

Set it up as a usual feed published table with an appropriate index on feed_sync_id.
Either alongside or within the cache, represent the latest loaded feed_sync_id.
Set up a cronjob/etc that reads the latest feed_sync_id and compares it to the cache’s feed_sync_id.
If they differ, reload the cache.
Ensure that all changes set feed_sync_id to null!

This works really well because the feed_sync_id in the database only gets updated on changes, so the reload cronjob mostly is a no-op. This means we can reload very frequently!

July 11, 2018

robustness principle and mocks

The Robustness Principle (or Postel’s Law) states

Be conservative in what you do, be liberal in what you accept from others (often reworded as “Be conservative in what you send, be liberal in what you accept”).

This principle has some criticisms.

I realized this has interesting implications for mocks. Suppose you have

public class MyObj {
  private final Integer x;
  
  public MyObj(Integer x) {
    if (x < 0) {
      throw new IllegalArgumentException("negative x!");
    }
    this.x = x;
  }
  
  public int getX() {
    return x;
  }
}

In a unit test, we can have

July 10, 2018

cross-dc sync with feed published KV

It’s been fun describing the feeds framework we use at Square. Today we’ll dive into a concrete problem:

We’ll stick with the feed-published kv table again.
We want two instances of some application code to bidirectionally synchronize the writes that happened on their instance to the other.
Eventually consistent is ok.

First, a bit of justification, though: I use this KV table to remember things I might have to look up without usual context, like my motorcycle’s license plate number, or that one weird python snippet I can never remember. I also have a whole slew of them at work – a bunch of random representative IDs for a bunch of things in our systems that I use from time to time. I also use a bunch of these as todo items at work, but that happens to work differently and is a topic for a future blog post :-)

July 2, 2018

history preserving data models

Start with a super simple data model:

CREATE TABLE kv (
  id BIGINT(22) NOT NULL AUTO_INCREMENT,
  k VARCHAR(255) NOT NULL,
  v LONGBLOB NOT NULL,
  PRIMARY KEY (`id`),
  UNIQUE KEY u_k (`k`)
) Engine=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin

Suppose we want to audit “changes” to this data model.

Approach 1: `kv_log`

add data model like

CREATE TABLE `kv_log` (
  id BIGINT(22) NOT NULL AUTO_INCREMENT,
  changed_at TIMESTAMP NOT NULL,
  k VARCHAR(255) NOT NULL,
  old_v LONGBLOB NOT NULL,
  new_v LONGBLOB NOT NULL,
  
)

current value query: unchanged

June 29, 2018

mysql feeds

At work, we use a pattern called feeds that gets an incredible amount of work done. I’ve been wanting to describe it here for quite a while, and now seems as good of time as any.

The basic premise is: You have a service A with some data that other “consuming” services B, C, and D want to find out about. Maybe the data is payments, maybe it’s support cases, maybe it’s password changes… whatever. The other services might include your data warehouse, some event listeners, whatever.

May 31, 2018

constraint violations in tests

I had some test code that looked something like

@Test public void a() {
  db.insert(0, "a");
  assertThat(db.query(1).getKey()).isEqualTo("a");
}

@Test public void b() {
  db.insert(0, "a");
  db.insert(1, "b");
  assertThat(db.query(2).getKey()).isEqualTo("b");
}

@Test public void c() {
  assertThat(db.query(3)).isNull();
}

There’s some stuff to like about this:

pretty straightforward which test is touching which records
pretty clear what you expect to see in the database.

There’s a bunch of coincidences here that make this work, though:

May 15, 2018

multiple thread test cases in java

Some of my day job lately has been trying to get a better handle on threads in Java. In particular, we had a really weird race condition resulting from multiple threads processing an input data stream and causing some unexpected conflicts in our data access layer.

To try to replicate the problem in tests, I started with something like this:

private static Runnable runWithDelay(String name, long millis) {
  return () -> {
    try {
      Thread.sleep(millis);
    } catch (InterruptedException e) {
      throw new RuntimeException(e);
    }
    System.out.println(name + " finished");
  };
}

This is just a Runnable that waits a while, and then prints that it finished. Then we can test with something like: