Drake 1.0.1: You Can Use It Now

We’re pleased to announce the release of Drake version 1.0.1. This release is a quick follow-up to the recently released Drake 1.0.0.

The 1.x version of Drake provides easy installation, including a Homebrew recipe for Mac users.

We’ve also included a commemorative logo upgrade, check it out!

Many thanks to @amalloy for his ownership, guidance, and leadership with Drake’s core codebase since he started at Factual.

Many thanks to @mavericklou for taking ownership of specific bugs & features and owning release management. Mav notably added a shiny new install/upgrade script, which makes it dead simple for Linux users to install and upgrade Drake. Mav also took over the Homebrew recipe for Drake and has been keeping it up-to-date.

Thanks to @chen-factual for recently contributing an honest-to-goodness missing feature to Drake, allowing us to better justify this 1.0.1 release. It’s not too hard to ask for a feature, but it takes brave and good people like Chen to actually release a feature.

And as always: A heartfelt thanks to everyone in the community who has contributed bug reports, feature requests, and most awesomely, code contributions and pull requests!

Some background on Drake and project versioning:

Drake (Factual’s data processing workflow tool) was first open sourced by us on January, 24, 2013. It was versioned 0.1.0 at that time. Over time we upgraded Drake and moderately increased the version tag. Two years in we considered Drake “barely famous” and found ourselves at version 0.1.6.

There’s a kind of unwritten rule of humility in the Clojure community around open sourced library versioning. Authors are reluctant to presume too much. As an example, the Leiningen build tool defaults a new project’s version to 0.1.0, but then some authors change that to 0.0.1 as a first step on a new project. Because, you know… let’s not be presumptuous.

Well, I was bragging to Alan about a Drake release, which I was calling 0.2.0. Alan rolled his eyes. He was like, “Considering the conventions put forth by Semantic Versioning, and considering that Drake has been in production at this company as well as other companies for years now, why not call it version 1 already?” I pushed back a bit, by surveying two existing open source Clojure projects I know of:

Aleph, Zach Tellman’s notorious asynchronous library for Clojure, is currently versioned 0.4.x. Keep in mind it’s been in production at Factual and other serious minded shops for years now. I asked Zach when he plans to cut a 1.0.0 and his reply started, “Each minor release has been effectively a ground-up rewrite…”.
Riemann, Kyle Kingsbury’s network event stream processing system written in Clojure, also enjoys prime time adoption in various production environments. And yet it’s versioned 0.2.x. I asked Kyle if he plans to cut a 1.0.0:

Kyle: Naw, I don’t think it’s 1.0 material yet. 🙂

me: What would it take to justify a 1.0 ??

Kyle: API stability, a bunch of protocol enhancements, dropping a bunch of deprecated stuff, performance stuff, bunch of bugfixes, disk persistence, maybe determinism & a better stream compiler…

me: when will the madness end !?

Kyle: Probably never? Or, like, we could say that a tool may be sufficiently stable for a job while not being finished yet. :-/</span>

Alan was copied on all these emails. He wasn’t convinced, and questioned my motives. “Chesterton’s Fence1!” was my last defense, but it didn’t stand (so to speak). In the end, Alan and Mav outvoted me 2-to-1 for Drake 1.0.0. Plus Artem (Drake’s original designer) was enthusiastic. So we presumptuously bumped Drake’s version to 1.0.0 and cut a release.

Of course I made sure to warn my colleagues: “You should most certainly not use this version of Drake! As a general rule, do not use a version x.0.0 of any product. Sorry, you’ll need to wait for Drake 1.0.1 or whatever…”.

As of today, we’re closing that loop. Drake 1.0.1 has been released, a.k.a. “Drake 1”. Please consider it officially usable and ready for production… if that’s not too presumptuous of us. We hope you like it.

Go make some great workflows!

– Aaron Crow, Factual engineer, Drake contributor

Notes

Thanks to Alan for teaching us about Chesterton’s Fence(s).

Drake after Two Years: “Barely Famous”

We released Drake (“Data workflow tool, like a ‘Make for data’”) two years ago. The impetus behind Drake was the classic “scratch your own itch”. We craved a text-based data workflow tool that would make it easier to build and manage our data processing tasks.

Extend Drake (Make for Data) with a Simple Clojure Project

Earlier this year we released Drake, an open source data workflow tool. It was exciting to see the interest in Drake. We were especially pleased by the quality of outside contributions made to the project (as one example: S3 support – thanks @howech!).

Introducing Drake, a Kind of ‘Make for Data’

Processing data can be a real a mess!
Here at Factual we’ve felt the pain of managing data workflows for a very long time. Here are just a few of the issues:

a multitude of steps, with complicated dependencies
code and input can change frequently — it’s tiring and error-prone to figure out what needs to be re-built
inputs scattered all over (home directories, NFS, HDFS, etc.), tough to maintain, tough to sustain repeatability

Paul Butler, a self-described Data Hacker, recently published an article called “Make for Data Scientists”, which explored the challenges of managing data processing work. Paul went on to explain why GNU Make could be a viable tool for easing this pain. He also pointed out some limitations with Make, for example the assumption that all data is local.

Factual Node.js Driver

We’re pleased to announce the release of the officially supported Node.js driver for Factual.
Get going!
You can get the driver using npm…
$ npm install factual-api
…and then install it in your project and connect:
var Factual = require(‘factual-api’);
var factual = new Factual(‘YOUR_KEY’, ‘YOUR_SECRET’);
(If you don’t have an API key, sign up to get one. It’s free and easy.)
Basic Query Example
Do a full-text search of the restaurant database for rows that match the terms “Coffee, Los Angeles”
factual.get(‘/t/restaurants-us’,{q:”Coffiee, Los Angeles”}, function (error, res) {
console.log(“show “+ res.included_rows +”/”+ res.total_row_count +” rows:”, res.data);
});
You can find out more about basic query capabilities in the docs for our Core API docs.
Geo Filter Example
The driver supports all of Factual’s Geo Filtering. Here’s an example of finding Starbucks locations near a latitude, longitude:
factual.get(‘/t/places’, {q:”starbucks”,
geo:{“$circle”:{“$center”:[34.041195,-118.331518],”$meters”:1000}}},
function (error, res) {
console.log(res.data);
});
Crosswalk Example
Factual’s Crosswalk service lets you map third-party (Yelp, Foursquare, etc.) identifiers for businesses or points of interest to each other where each ID represents the same place.

Factual Releases Drivers that Matter: Python, Clojure, Haskell

As Factual adds more data and features to its APIs, it’s important that developers have it easy. So we’ve put our heart into releasing client side drivers that matter. Yes, we’ve got drivers like the Java driver and the PHP driver and stuff like that, if that’s what you need. But we’ve also fielded a few drivers that matter in their own special ways:

Crosswalk the Web with Java and Factual

Factual is pleased to announce the release of a new Java driver for our API. The driver supports Factual’s API features, including Crosswalk, Resolve, and geo location functionality around the Factual Places API.