Factual began deploying Clojure to production in October of 2009. We used it cautiously and experimentally at first, confining ourselves to a narrow corner of our stack related to query interpretation.
As we experimented further with Clojure and applied it to more problems, we formed a very favorable impression of the overall language and related technologies. It’s true that Lisp is seen by many as a “weird” language, and to be sure, Clojure is not without its warts. On the other hand, if you’re looking for a fun and productive functional language that runs seamlessly on the JVM, Clojure brings massive value to the table.
Aaron Crow, one of Factual’s early Clojure advocates, has a nuanced take on the matter:
“Please don’t make me write any more Java.”
Before long, we discovered Cascalog, a Clojure-based query language that runs on top of Hadoop. For certain use cases, we found Cascalog to be a huge win over the alternatives, especially when considering clarity of code and developer productivity. Since Cascalog queries provide a higher level of abstraction than pure Java-based MapReduce jobs, there may be concessions made to performance, but the code is also much easier to understand and maintain.
Chun Kok, author of "Clojure on Hadoop: A New Hope", is philosophical about the trade-offs:
“Why not use Hive?”
Evan Gamble, our most seasoned Lisp veteran, applies Clojure to solve some of Factual’s more challenging Machine Learning problems. He explains his love of Clojure thusly:
“It’s still not as good as Common Lisp.”
Zach Tellman, creator of Aleph, is currently helping us build an engineering team in Factual’s Palo Alto office. Zach describes his seasoned, measured approach to using Clojure to conquer Factual’s complex engineering challenges:
“Yeah, I can hack that together this weekend.”
Of course, like every tool, Clojure is not a silver bullet and we don’t treat it as such. Our broader goal is to have a wide variety of tools in our toolbox, and carefully choose the best tool for each job. Boris Shimanovsky, Factual's Director of Engineering, clarifies the subtlety here:
“Have you shipped yet?”
Fortunately for those of us who love us some parentheses (and who doesn’t (right?)), Clojure is often chosen as the right tool for a project. We’re now using Clojure and related tech throughout our stack, including:
We’ve also fielded Clojure-based libraries for using Factual’s public API:
We’re becoming more confident in when and where it’s most appropriate to apply Clojure as a solution, and we’re excited about the possibilities and potential gains. Boris elucidates:
“Sometimes, the Lisp weenies are right.”
If you're an engineer and interested in helping Factual with hard stuff, check out our current openings.
We live and breathe data — some of us even think it is sexy. We believe the world would be a better place if more high quality data was made available, unencumbered and affordable. Better data equals faster innovation and better decisions.
The mission sounds simple, but we recognize the path to realizing it is a challenging yet rewarding one. What we set out to solve is difficult but critical to humanity—and that is the fuel that drives us to come to work every day.