Blog
The road to lasting experimentation
Helping iNNpulsa Colombia build experimental capabilities
17 June 2026
We work with many organisations to help them do more and better policy experimentation. But pushing an agency towards a Randomised Controlled Trial (RCT) from day one — before anyone agrees on what to measure and why, or before there is a clear path leading evidence to action — might get a trial implemented, but it does not develop the internal capacity that can run the next one without additional support. This is what we actually mean by building an experimental organisation: creating sustainable structures that outlast our collaboration. We do this not by imposing rigid frameworks, but by meeting agencies where they are.
Meeting agencies where they are
While A/B testing and RCTs are the gold standard of evaluation, it does not make sense to focus only on them if the organisation will not be able to continue their journey of experimentation once the support is gone.
This was the case with our recent 18-month collaboration with iNNpulsa, Colombia’s entrepreneurship and innovation agency, where we worked on their flagship Zasca programme. Zasca is iNNpulsa’s biggest programme to date, a nationwide intervention upskilling 16,000 formal and informal SMEs from six sectors across 89 support centres in multiple six-month cohorts. In this process, the programme was generating vast amounts of SME data at the application, programme start and finish stages; and collecting, managing and analysing that data was a daunting and urgent task for our counterparts at the iNNpulsa Analytica team. We had identified the right institutional function for policy experimentation — the Analytica team’s work on Zasca — , and we brought the experimental principles with us, but the time and space — the conditions — were missing; so as far as experimental methodologies were concerned, the RCTs would have to wait.
The pyramid approach to experimental methodologies
So we used a pyramid approach. Building on a strong foundation, every incremental step would provide value while building internal methodological capabilities. And we did so aided by SCALE, an evidence-driven tool to increase the effectiveness of business development support developed by the project funder, the Argidius Foundation.
Starting with a comprehensive theory of change, we mapped the needs, interventions, outputs and the expected short- and long-term outcomes of the Zasca programme, which hadn’t been fully fleshed out yet or had been kept implicit. This step provided us with the framework for the data and analysis prioritisation that followed, acting as a north star and a filtering mechanism.
Next, we focused on the data that was being collected and aggregated. This step required a two-pronged approach, one focusing on data quality and the other on data management.
The data that was being collected via questionnaires from programme participants was extensive — but not always useful. A considerable part was not aligned with the theory of change (nor was it an operational requirement), so it would become irrelevant at the evaluation stage. Further, the questions were confusing and not easily applicable to the types of enterprises and populations targeted by the programme. And yet, there were also questions missing! For instance, the programme structure included a whole stream of support focusing on the development of the self, yet nothing was being measured regarding self-efficacy. Working with the iNNpulsa team, the on-the-ground operators, and programme participants, we redid the questionnaires to maintain evaluation strength and reduce administrative burden by cutting up to 30% of the questions.
In parallel, we worked to automate the data pipelines and cohort reports, reducing the burden at the aggregation stage and ensuring that the reports themselves reflected the theory of change and were individually insightful. This upskilled our counterparts at the iNNpulsa Analytica team in LLM-supported automation and sped up the report-writing process from hours to minutes (an estimated efficiency gain of 75%). This freed up capacity on their end to take a step back from the everyday work on the programme and think broader about what else could be done to improve learning and increase impact. We had just strengthened the methodological foundations and created the institutional conditions for experimentation.
At that stage, we could focus our attention on the use of quasi-experimental methods — like differences-in-differences and instrumental variables — to evaluate the causal effects of the programme. This approach allowed us to leverage existing administrative data and make stronger causal claims than the descriptive evaluations to date. As a by-product of this process, we created a new methodology to geospatially locate the (often informal) SMEs in the Colombian territory, allowing iNNpulsa to gain a new understanding of their concentration and distribution, and us to evaluate whether the placement of a Zasca centre makes a difference for local growth (it does!). This work also led to an initial evaluation of the first cohorts of Zasca of roughly 3000 participants, which was presented mid-way through the programme to institutional and political stakeholders and was pivotal to ensure the continuation of a programme that had now been proven effective, given the existing data.
It is at this point that we could focus on robustly testing causal assumptions by using randomised controlled trials. Once the theoretical foundations were strong, they had been linked to high-quality, easy-to-access and easy-to-evaluate metrics, and we had squeezed all possible analysis out of the existing data. So we focused on upskilling the iNNpulsa Analytica team in experiments using our Impact Accelerator methodology to generate three experimental pilot designs for the Zasca programme: one focused on the effects of the combination of technical and financial support to Zasca recipients, one to increase post-programme follow-up response rates, and one to enhance the one-on-one support provided by operators through the introduction of syllabus recommendations based on the systematic analysis of participant needs using all of the data provided at the application and programme start stages. By creating capacity within the iNNpulsa Analytica team and increasing their experimental capabilities, we could anticipate the programme’s needs and develop experimental designs that felt like the next logical step for the organisation. And while iNNpulsa has undergone significant restructuring in recent months, which limits their ability to execute on those experiments, these ideas have now been picked up by other organisations in the Colombian ecosystem to test within their purview.
Ultimately, the questions we were able to answer through our collaboration were only the ones directly relevant to iNNpulsa, but the vast and novel data collected on the understudied informal SME sector can help answer many more. This is why we are also currently working to make the Zasca data accessible to researchers via our Research Network, so it can contribute to broader learnings for the Colombian ecosystem and beyond.
The long-term view on experimentation
None of this was fast, and not all of it was comfortable. Asking a team to redesign its own questionnaires, to formalise a Theory of Change that had been implicit for years, or to wait for the foundations before reaching for the method everyone associates with rigour, takes a particular kind of patience, from us and from them. Real progress here looked less like a single trial result and more like a team that could explain its own programme logic, trust its own data, and ask its own next question. Our major success was helping the iNNpulsa Analytica team professionalise how they think about impact, providing them with knowledge and structures that will outlast our collaboration and can be reproduced in future programmes.
Building this kind of capability takes longer than running a single experiment. But it is the difference between investing for the long term and chasing a short-term result: spend the time now on the foundations, and the organisation is left with a system for turning its own evidence into decisions, long after any one trial, or any one partner, is gone.