Reimagining Program Development and Evaluation

The Framework was truly transformative for us on many levels. It has impacted all aspects of how we approach evaluation–our assessment tools, how we collect data, when we collect data, and how we use it.

O IDEAS Impact Framework foi extremamente útil para orientar o projeto e a melhoria contínua do Programa BEM. Dois conceitos foram vitais. O primeiro foi saber que aprenderíamos com o que funcionou e o que não funcionou…. O segundo foi o princípio da iteração de ciclo rápido. Foi a primeira vez que a equipe aplicou uma metodologia baseada em testes, aprendizado e refinamento contínuo que nos possibilitou inovar dentro do prazo e do orçamento.

The IDEAS Impact Framework was extremely helpful to guide the design and continuous improvement of Programa BEM. Two concepts were vital. The first was the notion that we could learn not only from what worked, but also from what didn’t work… The second was the principle of fast-cycle iteration. It was the first time that the team applied a methodology based on testing, learning, and refining rapidly, and it enabled us to innovate on time and on budget.

Over the course of our 22-year-old history, Raising A Reader has had several independent evaluations demonstrate the success of our Classic Red Book Bag program and its impact on improving and sustaining home literacy environments. Through our work with the IDEAS Framework, however, we recognized that our theory of change was missing large components of what we do, such as community capacity-building and professional development in early childhood development…. With our updated theory of change, we are now better poised to implement the other components of the IDEAS Framework.

O IDEAS Impact Framework foi uma abordagem metodológica essencial para o desenvolvimento do programa Fortalecendo Laços, desde a ideação de como resolver um problema até a solução por meio da criação de um programa baseado em evidências científicas…. No início, tivemos que mudar a mentalidade para entender melhor o IDEAS Framework e como incorporar seus conceitos em nosso projeto… Esse desafio foi muito fácil de enfrentar porque o IDEAS compreende uma estrutura clara e definições operacionais precisas, permitindo-nos melhorar o cuidado metodológico do nosso estudo e a interpretação de nossos achados.

The IDEAS Impact Framework was an essential methodological approach for developing the Strengthening Bonds program –from the early ideation to solve a problem, until the solution of creating a scientific evidence-based program… In the beginning, we had to change the mindset to better understand the IDEAS Framework and how to incorporate its concepts into our project… This challenge was very easy to face because IDEAS has a clear structure and precise operational definitions, allowing us to improve the methodological care of our study and the interpretation of our findings.

Although the TOP strategies, targets, and outcomes were already developed, placing them in this new model (theory of change) gave us more insight into the mechanisms of the TOP Program…. The ToC and fidelity tool have also become the framework for our educational programs. This helped our interventionists to get better insights in how to transfer the key intervention strategies into practice.

IDEAS nos obligó a ir al campo de trabajo, a escuchar a las personas, observar, intuir, mirar el contexto a inspirarnos, en lugar de hacer todo el trabajo en el escritorio, con la tentación de reportar éxito y justificación de las acciones.

The IDEAS Framework brought us into the field, to listen to people, to observe, to intuit, to look at the context for inspiration, instead of doing all the work at our desks, with the temptation to report only the successes and to justify all our actions.