The politics of experimentation

(with Todd Hall)

Many startups have embraced formal experimentation—i.e., A/B testing—to improve the performance of their products and services. Experimentation, some have argued, should democratize innovation inside organizations by creating a platform for rigorously testing novel ideas, regardless of origin. In this article, we argue that experimentation’s promise hinges on having the right decision-making processes—one that encourages innovation but without the added risk of unanticipated failures. We study this question by developing a model of experimentation inside organizations where decisions to implement are either centralized or decentralized. We find that startups with centralized mechanisms benefit least from experimentation, as do ones with completely decentralized ones. In contrast, startups with mostly decentralized decisions, but with high standards for implementation, achieve growth with less downside risk. Thus, without considering the politics that guide organizational decision-making, the benefits of experimentation may be limited.

Educational technology and the problem of inequality

Work in progress

(with Anuj Kumar)

Innovations in educational technology are proposed by many as a solution to the growing achievement gap in America and abroad. Recently, however, there is increasing concern over whether scarce resources should be allocated to these yet unproven technological interventions—or spent on more proven inputs to educational production (e.g., better-trained teachers). Moreover, recent reports advance the hypothesis that technology may further increase inequality. In this article, we synthesize the literature to develop a framework for understanding the conditions under which technology leads to less, rather than more, equity in educational outcomes. Our analysis indicates that given the uneven distribution of essential inputs to learning—teachers, administration, and family resources—across schools and students of different socioeconomic status, educational technology interventions that ignore these differences will exacerbate existing achievement gaps, not mitigate them.

Idea uncertainty and founder-idea fit: Evidence from a field experiment

(with Aaron Chatterji)

A key hurdle faced by early-stage entrepreneurs is resolving the uncertainty around their business ideas. This challenge, scholars suggest, may lead to suppressed levels of entrepreneurial entry, or worse, the inefficient expenditure of effort on weak ideas—especially ones that a founder may be unsuited to execute. One way that entrepreneurs can overcome this friction is by getting expert advice, but prior work suggests such advice is often costly, idiosyncratic, and scarce. We investigate whether reducing the cost of idea evaluation will lead entrepreneurs to generate more and higher-quality ideas. We conducted a field experiment that provided early-stage founders with technology—a conversational A.I. platform to automatically research competitors, customers, and business models—to help them assess the market uncertainty of their ideas. We find that reducing the cost of evaluating market uncertainty leads entrepreneurs to generate more ideas, select those that are a better fit for their skills, and carry out more steps in the startup process. This finding is strongest for founders who are motivated by challenge and contribution, and less by money or status. Our results show that by reducing the cost evaluation, an important challenge faced by entrepreneurs can be overcome, especially in places where expert advice is scarce.

Hunting for talent: Firm-driven labor market search in America

(with Ines Black and Rembrand Koning)

What impact has the digitization of the labor market had on firm-driven search for talent? We investigate this question with two novel data sets. First, we conduct a nationally representative survey of American workers. We find that today over 18 percent of all employed workers in the US were hired into their present company by the outbound recruiting effort their employer, either directly or through labor market intermediaries such as a headhunter. The share of hiring driven by active firm search behavior is greatest among higher-income workers, at 23 percent, and those with STEM and business degrees, at 22.5 percent. Moreover, there is considerable regional variation. Over a quarter of Silicon Valley workers are hired in this manner. We complement our worker-level results with an analysis of a large sample of job postings in the US economy over the past decade. We find that firms, especially those relying on high-skilled labor, are increasingly developing capabilities to better hunt for talent. Given the prevalence of this practice, we discuss how theories of labor markets and firm strategy may be refined to account for these facts.

Designing social networks: Joint tasks and the formation and endurance of network ties

Journal of Organization Design, (9)4, 2020

(with Rembrand Koning)

Can managers influence the formation of organizational networks? In this article, we evaluate the effect of joint tasks on the creation of network ties with data from a novel field experiment with 112 aspiring entrepreneurs. During the study, we randomized individuals to a set of 15 joint tasks varying in duration (week-long teams to 20-minute conversations). We then evaluated the impact of these interactions on the formation and structure of individuals’ social networks. We find strong evidence that these designed interactions led to the systematic creation of new friendship and advice relations as well as changes to the participants’ network centrality. Overall, network ties formed after a randomized interaction account for about one-third the individuals a participant knows, of their friendships, and their advice relations. Nevertheless, roughly 90% of randomized interactions never become social ties of friendship or advice. A key result from our research is that while joint tasks may serve to structure the social consideration set of possible connections, individual preferences strongly shape the structure of networks. As a consequence, there will likely remain a considerable unpredictability in the presence of specific ties even when they are designed.

Experimentation and startup performance: Evidence from A/B testing

Available on SSRN

(with Rembrand Koning and Aaron Chatterji)

Recent work argues that experimentation is the appropriate framework for entrepreneurial strategy. We investigate this proposition by exploiting the time-varying adoption of A/B testing technology, which has drastically reduced the cost of experimentally testing business ideas. This paper provides the first evidence of how digital experimentation affects the performance of a large sample of high-technology startups using data that tracks their growth, technology use, and product launches. We find that, despite its prominence in the business press, relatively few firms have adopted A/B testing. However, among those that do, we find increased performance on several critical dimensions, including page views and new product introductions. Furthermore, A/B testing is positively related to tail outcomes, with younger ventures failing faster and older firms being more likely to scale. Firms with experienced managers derive more benefits from A/B testing. Our results inform the emerging literature on entrepreneurial strategy and how digitization and data-driven decision-making are shaping strategy.

Prior social ties and the limits of peer effects on startup team performance’

Forthcoming in Strategic Management Journal

(with Rembrand Koning)

We conduct a field experiment at an entrepreneurship bootcamp to investigate whether interaction with proximate peers shapes a nascent startup team’s performance. We find that teams whose members lack prior ties to others at the bootcamp experience peer effects that influence the quality of their product prototypes. A one-standard-deviation increase in the performance of proximate teams is related to a two-thirds standard-deviation improvement for a focal team. In contrast, we find that teams whose members have many prior ties interact less frequently with proximate peers, and thus their performance is unaffected by nearby teams. Our findings highlight how prior social connections, which are often a source of knowledge and influence, can limit new interactions and thus the ability of organizations to leverage peer effects to improve the performance of their members.