Programming on traviscj/blog

Transacter and Intent/Result

Wed, 15 Jun 2022 08:07:47 -0500

At SQ, we had a family of Transacter interfaces:

public interface Transacter {
  void transaction(Function<Session, Void> session);
  
  public interface Session {
    DSLContext dsl();
  }
}

where DSLContext is a jOOQ concept & is the handle for doing work against the database. Then this would be used in some class like

public class KvDao {
  @Inject Transacter transacter;
  
  void put(String ns, String k, String v) {
    transacter.transaction(session -> {
        KvRecord record = new KvRecord(ns, k, v);
        session.dsl().insertInto(KV).set(record).execute();
    });
  }
}

The parameters ns, k, and v are shorthand for namespace, key, and value, respectively. We’ll show how these might be used together shortly!

interceptors

Mon, 18 May 2020 15:13:00 -0700

INTERCEPTORS ARE SO COOL!

Sometimes you need some “generic rails” that are still highly adaptable to other uses. This is the basic problem solved by the Interceptor pattern.

I really love the way OkHttp does interceptors for the generic rails of making HTTP calls, so I wanted to walk through a case study of why an interceptor might be useful and then try to synthesize some lessons & a minimal example of the pattern.

smart arguments

Thu, 21 Feb 2019 00:00:00 +0000

It’s easy to love the terseness one can get out of a command-line tool with a “pass by order” convention. For example, maybe I run something like

deploy myapp sea2 production

to deploy an application called myapp to the second production datacenter near Seattle. This works great, except now I need to remember the order of arguments is: deploy [app] [datacenter] [environment].

The typical solve for this problem is introducing named arguments, so you’d end up with

pattern: extract mutating variables to State class

Mon, 07 Jan 2019 00:00:00 +0000

I’ve come to really like the pattern of extracting local or instance variables into their own State class, and writing an algorithm in terms of that State.

A really trivial (if contrived) example of this would be a “sum cubes” algorithm; something like this:

public class CubeSummer {
  private final PrintStream printStream;
  private long i;
  private long sum;
  
  public CubeSummer() {
    this.printStream = System.out;
    reset();
  }
  
  public long sumCubes(long limit) {
    for (; i<limit; i++) {
      sum += i * i * i;
      display();
    }
    return sum;
  }
  
  public void reset() {
    i = 0;
    sum = 0;
  }
  
  public void display() {
    printStream.print("i = " + i + ";");
    printStream.println(" sum = " + sum);
  }
}

What’s happened here is we’ve essentially promoted some variables that should arguably be local variables into instance variables, to avoid needing to pass them around explicitly. But in the process, we’ve also mixed long-lived state with short-lived state, broken thread safety, and forced clients of this class into a particular calling convention (either instantiate an instance per sumCubes call or carefully respect thread safety and call #reset().)

robustness principle and mocks

Wed, 11 Jul 2018 11:53:54 +0000

The Robustness Principle (or Postel’s Law) states

Be conservative in what you do, be liberal in what you accept from others (often reworded as “Be conservative in what you send, be liberal in what you accept”).

This principle has some criticisms.

I realized this has interesting implications for mocks. Suppose you have

public class MyObj {
  private final Integer x;
  
  public MyObj(Integer x) {
    if (x < 0) {
      throw new IllegalArgumentException("negative x!");
    }
    this.x = x;
  }
  
  public int getX() {
    return x;
  }
}

In a unit test, we can have

constraint violations in tests

Thu, 31 May 2018 00:00:00 +0000

I had some test code that looked something like

@Test public void a() {
  db.insert(0, "a");
  assertThat(db.query(1).getKey()).isEqualTo("a");
}

@Test public void b() {
  db.insert(0, "a");
  db.insert(1, "b");
  assertThat(db.query(2).getKey()).isEqualTo("b");
}

@Test public void c() {
  assertThat(db.query(3)).isNull();
}

There’s some stuff to like about this:

pretty straightforward which test is touching which records
pretty clear what you expect to see in the database.

There’s a bunch of coincidences here that make this work, though:

regex in java

Fri, 23 Feb 2018 13:41:22 +0000

I can never remember how to do java regexes with Pattern.

Pattern pattern = Pattern.compile("x_(yy|zz)");
String input = "x_yy";
boolean m = pattern.asPredicate().test(input);
System.out.println("matches: " + m);
Matcher matcher = pattern.matcher(input);
while (matcher.find()) {
  System.out.println("whole matched string: " + matcher.group(0));
  System.out.println("matched group: " + matcher.group(1));
}

I also learned a really cool thing IntelliJ can do with pattern matches!

![java regex]({{ site.baseurl }}/assets/java-regex.png)

guava RangeMap

Fri, 26 Jan 2018 09:52:41 +0000

We had some code like this:

NavigableMap<Double, Integer> PRIORITY_MULTIPLIERS = new TreeMap<Double, Integer>() {{ "{{ " }}
  put(LOW_THRESHOLD, 1);    // (20k..40k)  => 1
  put(MEDIUM_THRESHOLD, 2); // [40k..100k) => 2
  put(HIGH_THRESHOLD, 3);   // [100k..+∞)  => 3
}};

Integer getPriorityMultiplier(Double v) {
  if (v < LOW_THRESHOLD) {
    return 0;
  }
  return PRIORITY_MULTIPLIERS.floorEntry(v).getValue()
}

I replaced with a [RangeMap][RangeMap] that uses [Range][Range] instances as keys: [RangeMap]: https://google.github.io/guava/releases/16.0/api/docs/com/google/common/collect/RangeMap.html [Range]: https://google.github.io/guava/releases/16.0/api/docs/com/google/common/collect/Range.html

RangeMap<Double, Integer> PRIORITY_MULTIPLIERS = ImmutableRangeMap.<Double, Long>builder()
      .put(Range.lessThan(LOW_THRESHOLD), 0)
      .put(Range.closedOpen(LOW_THRESHOLD, MEDIUM_THRESHOLD), 1)
      .put(Range.closedOpen(MEDIUM_THRESHOLD, HIGH_THRESHOLD), 2)
      .put(Range.atLeast(HIGH_THRESHOLD), 3)
      .build();
Integer getPriorityMultiplier(Double v) {
  return rangePriorityMultipliers.get(v);
}

Advantages here being:

todobackend.com

Fri, 05 Jan 2018 00:01:00 +0000

TodoBackend.com defines a simple API and has a frontend that works given any backend URL.

Particularly interesting to me:

haskell
- super concise!
- whoa, 6 different backends!
- spock backend is crazy concise.
python + flask
- seems like it would greatly benefit from a Store object
scala + akka
- crazy pattern matching in routes definition
java + mangooio + jOOQ
- especially the distinction between Todo and TodoPatch
- also really glad I don’t have to write routes config separately from the code!
clojure + jetty/compojure + postgresql
rails
go + goa

url encoding safe relational operators

Wed, 29 Nov 2017 07:55:04 +0000

I was catching up on Jesse Wilson’s blog this morning, and in particular his post URL Encoding Is Material. In that article, he says

My advice

If you’re defining your own URLs, you’ll save a lot of trouble by avoiding characters like <, >, {, }, +, ^, &, |, and ;.

I came across this problem in one of my projects at work: I wanted a page to display some recent records with 1) a name field provided by the user and 2) a value field that match $l \leq \text{value} \leq b$, where $l$ and $b$ are given by the user. I also wanted that page to have a URL representing that condition, so that it could be easily shared with teammates and soforth. When I started working on it, I didn’t have Jesse’s wisdom, and so I naively came up with a URL structure something like

python attrs

Thu, 31 Aug 2017 08:42:43 +0000

I came across a very interesting library in a HN thread: the python attrs library.

In particular, this seems like a great way to do the “dumb data objects” they talk about in the end of object inheritance, and also related to (but maybe lighter weight than) zope.interface.

This also seems very similar to what I use autovalue for at work.

One particularly interesting application is a “code database” – using static, checked-in-to-version-control definitions of some data model as a sort of very-fast-to-read, very-slow-to-update “Data Model”. I find this fascinating: Code shares a lot of properties with great data stores: ability to rollback (git revert) and accountability/auditability (git blame). It also makes a lot of fairly hard problems much simpler: you don’t need to poll the database for changes. You don’t need to invalidate any caches. You don’t need to consider a “split brain” environment where half of the in-memory caches have updated but the other half haven’t. You don’t need to consider failure cases of how long the in-memory cache is allowed to be invalid: you just fail to boot up on deploy. (Admittedly, there’s still an opportunity window for split brain behavior for the duration of the deploy, but this is a lot easier to reason about than an essentially arbitrary.)

use the `locale.currency` library function

Fri, 20 Jan 2017 00:00:00 +0000

I had stupidly written a pair of functions like this:

def comma_sep(amount):
    thousands = int(amount/1000)
    remainder = int(amount%1000)
    if thousands > 0:
        return comma_sep(thousands)+","+"%.3d"%(remainder)
    else:
        return str(remainder)
def friendly_from_cents(total_cents):
    pos = total_cents >= 0
    total_cents = abs(total_cents)
    cents = int(total_cents%100)
    dollars = int(total_cents/100)
    friendly = "$"+comma_sep(dollars)+"."+str(cents)
    if pos:
        return friendly
    else:
        return "-" + friendly

This is stupid, because python has builtin support to localize currency:

import locale
locale.setlocale(locale.LC_ALL, 'en_US')
locale.currency(-1800, grouping=True)

I think it’s tempting to fall into these traps, because it always starts simpler: I had some total_cents variable and just wanted to display quick and dirty. The complexity built up gradually, until I had something pretty complicated. Oops.

filter vs spec (draft)

Wed, 18 Jan 2017 00:00:00 +0000

Consider a silly data model to store data about cities like

message City {
  optional string city_name = 1;
  optional string state = 2;
  optional int32 population = 3;
  optional int32 year_founded = 4;
  // ... presumably others :-)
}

and some sample data like:

[
  {"city_name": "Portland", "state": "OR", "population": ...},
  {"city_name": "Portland", "state": "ME", "population": ...},
  {"city_name": "Springfield", "state": "FL", "population": ...},
  {"city_name": "Springfield", "state": "IL", "population": ...},
  {"city_name": "Springfield", "state": "CO", "population": ...}
]

There are some useful entities we can define: (DRAFT NB: don’t read too much into the matcher vs filter lingo.)

CBCL: The Common Business Communication Language

Mon, 12 Dec 2016 23:05:28 +0000

I recently came across McCarthy’s CBCL paper on Hacker News. He presents a Lisp notation as a format for sharing requests between different software on different computers. He calls out

(PLEASE-RESERVE (EPSILON (x) 
  (AND
    (IS-FLIGHT x) 
    (DEPARTS MONDAY) 
    (ARRIVES (BEFORE WEDNESDAY)))))

where $\epsilon x$ is an operator maps a predicate to a value matching the predicate, and considers it an improvement on the “iota” definite description operator, which seems to be an operator that returns the unique value matching the predicate. It seems interesting to me that the unique match operator is not considered as useful as the arbitrary match operator – unique keys actually seem critical to a lot of my work with data models. Even more fascinating, though, is that this looks like an anonymous function!

oblique programming strategies

Wed, 07 Dec 2016 23:05:28 +0000

Ever since I found out about it (probably on Hacker News), the idea of Oblique Strategies has fascinated me. The first editions are going on ebay for $2500-$3300 bucks, which I think is incredible. If you’re curious and impatient, you can check out this list on github.

One recent sleepness night, I made a list of “oblique programming strategies” on my phone, transcribed here. They are not as starkly polished as Eno’s version (unsurprisingly), but might be useful to you!

logging

Fri, 26 Sep 2014 00:00:00 +0000

In grad school, I spent a lot of time writing code that read output from nonlinear optimization solvers, and tried to do useful things with it. A much better way to do that is called “structured logging”, an idea I experimented with a bit during grad school. It has also been coming up in my working life, so I wanted to delve into it a bit deeper. For a quick introduction, check out Thoughts on Logging. For a lot longer introduction, see The Log: What every software engineer should know about real-time data unifying.

language fluidity

Sun, 29 Dec 2013 00:00:00 +0000

I have learned a lot of programming languages over the course of my life. So I wanted to do two things:

Discuss a few effective ways to pick up a new language.
Discuss a few references that I’ve found helpful for some languages. I should start with a disclaimer: I do enjoy learning new languages, but mostly I enjoy creating things in those languages. I’m not sure all the effort I have spent learning many languages would not have been better spent learning many fewer languages to a deeper level, but I do feel like it is extremely important to learn principles, not languages, and to use the best tool for a particular job. Many times principles are most clearly illustrated with contrast between languages, and many more times a particular tool has been a patently poor choice for a particular job.

That being said, it’s important to know your “first tongue” languages in significant depth.

square

Sat, 16 Nov 2013 00:00:00 +0000

I probably should have been working on my thesis, but I had a phone interview with Square. I wanted to write a bit of code to get myself “warmed up” but wanted it to be kinda fun. So I decided to write some code that drew a square.

#include <iostream>
using namespace std;

int main() {

    cout << "+";
    for (int i=0; i<8; i++) cout << "-";
    cout << "+" << endl;

    for (int j=0; j<5; j++) {
        cout << "|";
        for (int i=0; i<8; i++)
            cout << " ";
        cout << "|";
        cout << endl;
    }

    cout << "+";
    for (int i=0; i<8; i++) cout << "-";
    cout << "+" << endl;

    return 0;
}

which yields a nice little square:

overly-ambitious-isqo and the design of numerical codes

Wed, 23 Oct 2013 00:00:00 +0000

I have finally released the overly-ambitious-isqo project on github!

I wanted to call out two particular design concerns I had.

rich language

My first goal was to try very hard to build up the C language to very succinctly express the main algorithm in src/isqo_functor.cpp in extremely rich language. It seems like numerical code is typically implemented with loops like for (int i=0; i<N; i) and method calls like deltal(xhat, mu). I have found it much easier to reason and think deeply about codes like for (int primal_index=0; primal_index < num_primal; primal_index) and method calls like linear_model_reduction(penalty_iterate, penalty_parameter).

C/C++ Values, Pointers, and References

Fri, 20 Sep 2013 00:00:00 +0000

I had an embarrassing realization: While C does have an address-of operator, it does not include reference variable or pass-by-reference syntax. It’s embarrassing because I should have realized that it was a C++ invention; I guess some of my “C” code actually really has been C++ code.

For future reference, here’s the current summary of my understanding:

	var	*var	&var
int xv=3;	value	invalid	memory location of value
int *xp=&xv;	memory location	value	memory location of pointer.
int &xr=xr;	value	invalid	memory location of value	(all invalid in C)

As one other note, there are three ways you can pass things in C:

python decorators

Tue, 30 Jul 2013 00:00:00 +0000

I came across something that I think is actually pretty neat. Python supports ``decorating’’ functions, which lets you define behaviors that essentially wrap around a given function. The benefit is that you can write a simple function and decorate it to do something fancier.

I realized I wanted to do this with a simple timing output. Say my function was just something really simple like

def f_plain(x,y):
    print("x + y = {}".format( x+y))
    return x+y
print ("---")
print (f_plain(1,3))

Now, to time it, I could define a function like:

implementation of set operations

Wed, 13 Mar 2013 00:00:00 +0000

We got in a bit of a debate yesterday in the office over the implementation of associative containers, which I thought was pretty fun. We made up the big chart of complexity results you see below.

nomenclature:

$S$, $S_1$, and $S_2$ are subsets of $\Omega$.
Denote an element by $e\in\Omega$.
$n$,$n_1$,$n_2$,$N$ are the sizes of the set $S$, $S_1$, $S_2$, and $\Omega$, respectively, and $n_1 \geq n_2$.

Complexity

Operation\Approach	Hash Table	Hash Tree	Binary List	Entry List (sorted)	Entry List (unsorted)
$e \in S $	$O(1) $	$O(log(n)) $	$O(1) $	$O(log(n)) $	$O(n) $
$S_1 \cup S_2 $	$O(n_1+n_2) $	$O(n_1+n_2) $	$O(N) $	$O(n_1+n_2) $	$O(n_1n_2) $
$S_1 \cap S_2 $	$O(n_1) $	$O(log(n_1)n_2)$	$O(N) $	$O(n_2) $	$O(n_1n_2) $
space complexity	$O(n) $	$O(n) $	$O(N)$ bits.	$O(n) $	$O(n) $

As I said–this was just what came out of my memory of an informal discussion, so I make no guarantees that any of it is correct. Let me know if you spot something wrong! We used the examples $S_1 = {1,2,3,4,5}$ and $S_2 = {500000}$ to think through some things.

some tips for project euler problems

Sat, 26 Jan 2013 00:00:00 +0000

I wanted to make a list of a few tips on solving Project Euler problems that have been helpful for me while I solve them. These are general principles, even though I do most of my Project Euler coding in Python.

Without further ado:

If the problem is asking for something concerning the number of digits, typically this indicates that the use of the $\log n$ function is warranted.
If the problem is asking for the last few digits, modulo arithmetic might speed it up considerably.
Some might consider this cheating, but looking up some small numbers in the Online Encyclopedia of Integer Sequences is occasionally pretty helpful.
Many problems boil down to: Find numbers with property $X$ and property $Y$. Two solutions are:
- Brute force: Try all numbers with tests of property $X$ and $Y$.
- Find numbers with property $X$ and filter by a test of property $Y$.
- Find numbers with property $Y$ and filter by a test of property $X$.
- Find the set of numbers with property $X$ and the set of numbers with property $Y$. Compute their intersection.

I’ve found that it’s sometimes hard to predict which one will end up being the fastest. It depends on the relative speed of the tests and the generators, and the frequency of finding numbers which have that property.

Integrating Sparrow and Aperture/iPhoto

Fri, 13 Apr 2012 00:00:00 +0000

In my recent quest to take on digital photography, I’ve been using Apple’s Aperture to store download my pictures and do basic manipulation on them. It’s been working great, except that emailing is a pain because Apple hardcoded the email clients to:

Apple Mail.app
Eudora (really!?)
America Online(!?!)
Microsoft Entourage
Microsoft Outlook

while I’d prefer to use Sparrow, a lightweight client that Sharvil got me hooked on a while back. Rather than continue to use Mail.app to send pictures via email, I found out that Aperture actually uses some AppleScripts to do the actual sending of the email. That got me thinking: I just need to change the script to mail using Sparrow, instead of Mail.

HMCP Set, TextMate, and RoR 2.0

Fri, 06 Feb 2009 00:00:00 +0000

I found a great little LaTeX homework layout for typesetting math homeworks a while back, and I’ve meant to write it up for quite a while. It’s from Harvey Mudd College’s Mathematics Department, engineered to automatically satisfy the department guidelines on how homeworks should be laid out. They also have thesis and poster templates, but the homework installation proceedures and sty files are located on the Homework Class page. Their website also has extensive information on their computer resources. While basically useless for anyone not at Harvey Mudd, it is a model for how this sort of website should look. Everything is clearly detailed at the perfect level, along with suggestions not just for how to do certain tasks, but also when you might want to do something.

Fame and FORTRAN

Sat, 08 Nov 2008 00:00:00 +0000

I must be getting more popular on some search engines somewhere. I just got six random comment-spam messages. Awesome. I guess that’s why the more important bloggers have come to rely on Bayesian filters and soforth for taming the wild flow of spam. Hopefully that trend doesn’t continue.

Also, it seems as though I am now learning FORTRAN. I’m sortof starting working with Eric Shea-Brown on some Neuroscience research, working with HPC on NSF’s Teragrid. It’s pretty exciting stuff, and I’m really excited about getting moving on it. Anyways, back to FORTRANizing, I suppose.

PHP Vulnerability Checklist

Thu, 10 Jul 2008 12:00:00 +0000

Ray(the boss) and I were chatting about how we should implement our secure upload page. The way it should work is, an IP is added to the genhosts database, it sets up a 5 minute wait, and at the end of five minutes, redirects to the data upload page, at which point the computer is allowed to connect, since the new IP has propogated into /etc/hosts.allow(via a cronjob that takes MySQL entries and parses them(via HTTP and XML) into the standard /etc/hosts.allow format). It’s basically working, but we’re having a bunch of trouble getting IE and Firefox to redirect, since IE apparently detects my window.location.href and location.replace calls as popups, and blocks them.

Cython

Fri, 09 May 2008 12:00:00 +0000

After I had finally convinced myself to get out of bed this morning to go to my ACMS seminar, I quickly checked my email and my heart sank a little. Today’s talk was on SAGE. Don’t have anything against SAGE, but I thought it was just a big pile of open source packages in a big, heavy install. Sorta cool, but worthless, in other words.

Turns out, I was pretty wrong about that. It is that, but it’s also 70k new lines of code that does a whole bunch of exciting stuff. Near the end of his talk, William Stein mentioned that they had created a new tool called Cython. (Well, extended Pyrex, but… whatever.)

Python Playing

Fri, 28 Dec 2007 12:00:00 +0000

I’ve been playing around with Python a bit more over the break, mostly because I want to be lazy in my code-writing for the forseeable future, plus I’d like to give NumPy a shot, but I want to have a solid background before trying that. One thing that just has not gotten old yet is the ability to assign functions absolutely anywhere in python. You can pass them around like variables, put them in dictionaries, return them from functions, whatever. I mean, I lived in parenthesis for a while in my scheme class, so it’s not like I haven’t been exposed to it, but it’s still damn cool. You can do cool stuff like making a tuple of some index, some value, and the function to process the other two. That makes it so much easier to deal with various combinations of neat datastructures. Anyways, the book I’ve been reading is Dive Into Python, off diveintopython.org. I’m currently at about page 114 out of 327, and I’ve fairly faithfully read most of it up until that point. The format seems slightly frustrating to me, but it’s probably just because it’s such a departure from what a lot of books and tutorials use. It seems helpful and worthwhile in any case.