We all have those conferences we dread every year—the same people in the same ties with the same boring ideas. This is not one of those conferences.
Recently we sent Brian Aitken and Anthony Chiaravallo, our creative strategy team from Aitken+Vallo, to the first ever CODE Con, the Conference on Digital Experimentation at MIT Sloan School of Management. With an exclusive VIP backstage pass to this first-of-its-kind academics only conference, our team had a chance to delve into the minds of Kellogg School of Management’s Eric Anderson, Economics Professor Harvard University Sendhil Mullainathan, Microsoft Research Director Eric Horvitz and other like-minded game changers.
The weekend focused on all new research being presented from the forerunners in digital experimentation during a unique format where each research session featured 45 minutes of open dialogue between presenters and attendees.
We had a chance to catch up with Sinal Aral, Associate Professor at MIT Sloan and one of the key players in seeing CODE Con to fruition. When discussing the practical applications for businesses of the theories discussed Sinan told us, “The ability to rapidly deploy micro-level randomized experiments at population scale is one of the most significant innovations for modern social and business decision making. Randomized experiments are the gold standard of causal inference and a cornerstone of effective policy. But the scale and complexity of these experiments also create scientific, statistical and ethical challenges for experimental design and policy.
The Conference on Digital Experimentation at MIT (CODE) brought together leading thinkers conducting and analyzing large scale randomized experiments in digitally mediated social and economic environments, in various disciplines including economics, computer science and sociology, in order to lay the foundation for ongoing relationships and to build a lasting multidisciplinary research community.
The community discussed the cutting edge of experimental research and practice and several companies like Airbnb, Facebook and Microsoft (among others) shared their current state of the art. Companies are implementing large scale experiments every day to improve decision making. The conference highlighted how these ideas can be put to use today to improve outcomes for consumers, businesses and for society as well. Business was not the only thing on the agenda. Attendees also discussed experiments to improve economic development, HIV testing, poverty alleviation, voting and other social policy goals.”
Eric Anderson from Kellogg presented a particularly fascinating abstract on Transforming Marketing Analytics in Consumer Focused Organizations. Opportunities in marketing analytics are moving much faster than most organizations can handle. As a result, firms have become reactive and tactical. In his talk, he explored how firms can adopt a more strategic, customer-focused perspective and said that predictive marketing analytics, driven by rapid experimentation, is central to this transformation.
Other talks like Sendhil Mullainathan’s focused on the fact that The Necessity for Causation is Overstated. As Sendhil put it, “To make empirically driven decisions, our methods must reveal causal patterns. We must know what outcomes each of our choices results in. As a result, experimentation is quickly becoming the tool of choice for decision making. In contrast, it appears that machine learning models because they only produce correlations are problematic for guiding choices. I argue that this is too narrow a perspective on decision-making, that there are many (important) decisions where predictions–with only correlation and no causation–are exactly what are needed for decision making. Understanding the dual role of prediction and causation in decision making will allow us to better combine machine learning and experimentation.”
The conference was definitely academic in nature, but highlighted many high-level strategic concerns for businesses today like utilizing micro-level randomized experiments at population scale to inform business decisions as well as ethical concerns businesses face with access to what can be highly personal, and sometimes sensitive, data. We wouldn’t be surprised if next years CODECon drew a much larger audience.
Speaking of which, next year’s CODE Con dates have yet to be announced but, we suggest you get on the waiting list.
