In the dynamic world of innovation funding, the strategic use of data is becoming indispensable. Recognising this, the European Innovation Council (EIC), in partnership with the Innovation Growth Lab (IGL), has embarked on an ambitious benchmarking study. The goal is to explore how innovation agencies across Europe are leveraging data to drive decision-making and strategic operations in order to improve the EIC’s standing in this context through the best available practices and peer learning exchanges.
Our findings indicate that agencies often tackle data challenges in isolation, limiting the potential for cross-agency learning and best practice sharing. The EIC, through its partnership with IGL, aims to transform this approach by fostering data-centric dialogues among agencies. This collaborative effort is designed to not only address individual agency needs, but also to elevate the overall standard of data-driven innovation funding across Europe.
Drawing inspiration from Francis Bacon’s analogy of ant researchers, spider researchers, and bee researchers, we can draw parallels with innovation agencies in their approach to data. Some agencies, like ‘data ants’, diligently collect data but often lack the mechanisms for deep analysis. Others, akin to ‘data spiders’, rely on experience and intuition rather than data collection, waving assumptions with scarce empirical support. And some agencies are yet to enter this world, who are neither actively gathering data nor engaging in analysis but are poised to evolve their practices.
We aim to guide these varied agencies towards becoming ‘data bees’, or entities that not only gather diverse data but also engage in thoughtful analysis and collaborative learning. Just as bees gather nectar from various flowers to produce honey, agencies can collect diverse data insights, learn from each other’s experiences, and collaboratively create a richer, more comprehensive understanding of the innovation landscape. This transformation, central to our study, shifts the solitary nature of data handling and analysis into a collective endeavour, enhancing the quality and impact of data-driven decisions across innovation agencies.
What we have found so far
Data utilisation has considerable untapped potential. The study revealed a consistent theme of underutilisation of data across agencies, attributed to limited capacity, skill mismatches, and a perceived lack of demand from decision-makers. Current efforts are primarily centred on financial and control data, with only a few agencies delving into the extensive use of data for measuring impact and monitoring projects. This underutilisation points towards a significant opportunity for agencies to harness their data more effectively.
Agency approaches to data projects are very diverse. We observed considerable variation in how data projects are requested, processed, and communicated across different agencies. Some agencies grapple with ad-hoc requests, while others have well-defined processes in place. Advanced request pipelines often emerge from bottom-up initiatives driven by motivated individuals, highlighting the importance of fostering a culture of innovation and proactive problem-solving.
Institutional communication needs and approaches vary greatly. Communication methods to policymakers vary, reflecting the differing needs and capacities of agencies. Agencies dealing with complex data may lean towards sophisticated dashboards and causal analysis, while others prefer narrative reports for their accessible nature. This variation highlights the necessity for data management and collaboration strategies to be adaptable, catering to the differing appetites and analytical needs of agencies.
Agency sizes shape data capabilities and risks. Agency size significantly influences data capabilities and challenges. Larger agencies often deal with more complex data needs and procedures; smaller ones benefit from more streamlined processes but face heightened risks associated with funding instability and loss of expertise due to reliance on individuals. This finding underscores the importance of developing robust data management strategies adaptable to the size and structure of the agency.
Siloed operations get in the way of data integration. A common challenge across agencies is the integration of disparate data sources. Factors contributing to this include a lack of joint ownership and collaboration, leading to siloed operations and data practices. Efforts to overcome these silos are crucial for agencies to leverage their data.
Ensuring data quality rather than availability is the challenge. Data availability is seldom a primary issue for agencies, but the quality of this data often emerges as a critical concern. Many agencies face challenges with missing or erroneous data. To address these issues, strategies range from quasi-random or ex-post error detection to intermediate analysis steps that identify and alert shifts in data distributions.
Limited data literacy curbs enhanced data utilisation. The study highlights a gap between the availability of data and analysis skills within agencies and their practical application. This gap is often influenced by the varying levels of familiarity with data analysis among decision-makers. Enhancing understanding and appreciation of advanced data analysis techniques, such as counterfactuals, experiments, and regressions, is crucial for agencies to make more informed decisions.
Data-driven decision-making is leading to more agile and efficient operations. Agencies are increasingly using data to refine their operational procedures. For example, one agency streamlined its application review process using experimental evaluations and enhanced grantee monitoring, a move towards more agile and evidence-driven approaches. The integration of external data is also supporting some agencies' ability to control for variations in proposal quality, helping standardise operational processes.
Bottom-up initiatives are catalysing agencies’ data evolution. Grassroots initiatives within agencies are significantly impacting data utilisation and management. These efforts reflect a growing motivation among staff to integrate data into operational decisions, indicating a shift towards data-driven decision-making, even in the absence of top-down directives.
What is to come in the future
Our benchmarking study with the EIC and various innovation agencies across Europe shows that the journey towards effective data utilisation is both complex and varied. It serves to highlight the diverse challenges and opportunities within these agencies but also sets the stage for a more unified and effective approach to data-driven innovation funding.
The insights gleaned from this study underscore the importance of moving beyond isolated data practices to a more collaborative model, akin to the ‘bees’ in Francis Bacon’s analogy. By fostering environments where agencies can share insights, learn from each other, and collectively enhance their data strategies, we can elevate the entire field of European innovation funding. This shift is crucial for agencies to fully harness the potential of their data.
For more insights and updates on this ongoing collaboration, we invite you to read our blog post about the project.
The individual benchmarking exercise will continue until the spring of 2024. If you want your innovation agency to participate and receive a report on the organisation’s position and potential, register your interest.
Sara Garcia Arteagoitia
Sara García Arteagoitia joined IGL as a Senior Researcher in 2022. In her role, she helps identify experimental research opportunities with our global government partners and bring them to fruition.
Before joining IGL, Sara worked as a consultant and instructor in behavioural design with private companies and government agencies in Spain and Latin America, helping them improve their structures and programmes to achieve higher impact. Prior to that, she was the Senior Assistant Director of the Center for Decision Research at the University of Chicago, where she worked on developing partnerships and research structures to propel and promulgate experimental behavioural research.
She started her career in experimental research as a research assistant for different projects at the University of Chicago and the Max-Planck Institute for Collective Goods. Sara holds a BA in Political Science and a MA in Political Psychology from the University of Heidelberg, Germany.
David Ampudia
David is a Data Scientist at the Innovation Growth Lab (IGL).
His role at the Lab focuses on helping grow the team’s capability for using data analysis and statistics to uncover and pursue opportunities to improve the effectiveness and equity of innovation and business policy.
Prior to joining the IGL in 2022, David worked as a Research Fellow at the Data Center of the Barcelona School of Economics (BSE). His work involved the development of state-of-the-art sequential decision-making algorithms, with a particular focus on Bayesian Optimization and multi-armed bandits. He also contributed to multiple political economy projects, helping develop and implement improved methodologies using novel data sources and techniques. As a member of the Center, David has taught text-based deep learning courses at the Summer School of the BSE for several years now. Before this, David worked as an economic consultant for FocusEconomics, a macroeconomic research and forecasting institute.
David holds a one-year MSc in Data Science Methodology from the Barcelona School of Economics, and a two-year MRes in Analysis and Policy in Economics at the Paris School of Economics. He also has a BA in International Economics from the Universitat Pompeu Fabra.