Ir a la descripción del puesto

E T Consultant

Department for Outcomes
Cierra el 23 nov 2026 · 3 días EST (UTC-5)
El empleador puede cerrarla antes.
Condiciones del lugar de destino
Haga clic en cualquier elemento para ver el detalle.
Descripción del puesto

At the World Bank, a member of the World Bank Group (WBG), you’ll join a diverse, global community working across cultures, disciplines, and borders to address the world’s most pressing development challenges. With 1.2 billion young people reaching working age in the decade ahead, the challenge of job creation has never been greater. Through partnerships across 145 countries and more than 182 offices worldwide, we work with governments, private sector, development partners and other stakeholders to invest in people, strengthen markets, and deliver scalable, data-driven solutions that generate more and better jobs and improve lives. For more information, visit International Bank for Reconstruction and Development VPU Context: The Managing Director and Chief Knowledge Officer (MDCKO) oversees the World Bank Group's knowledge and partnership agenda and helps ensure that the institution's analytical and operational work responds to the most pressing development challenges facing client countries and global partners. With the launch of the Knowledge Bank in 2026, the World Bank Group is deepening its focus on global knowledge on what works, where, and why. As part of this model, the World Bank Group now operates through five vertical vice presidencies aligned with the mission: People, Prosperity, Planet, Infrastructure, and Digital AI. In addition, three horizontal departments reporting to the MDCKO enhance knowledge, innovation, and outcome orientation across the World Bank Group. One of those horizontal departments focuses on improving the institution outcome orientation: Department for Outcomes. Department Context: The Department for Outcomes provides strategic vision and direction for stronger outcome orientation across the World Bank Group. Its mandate includes advancing thought leadership on development outcomes measure-ment, proposing revisions and updates to results indicators and methodologies for both project and portfolio-level management, coordinating the development of outcome targets, and strengthening how results are meas-ured and used across the institution. The Department manages the WBG Scorecard — the institution's apex out-comes reporting tool — and supports the development of metrics and systems to evaluate outcomes on cross-cutting themes such as climate, jobs, private sector capital mobilization, women, and youth. Within this broader mandate, the Department is advancing the AI for Outcomes program within the Outcomes Delivery Unit, which applies artificial intelligence and advanced data science to strengthen outcomes measure-ment, generate operational insight, and support evidence-based decision making across World Bank Group op-erations. The program has developed a suite of AI-powered analytical tools and applications now in active use, and ensuring their quality, reliability, and fitness for purpose is a growing institutional priority. Duties and accountabilities: The ET Consultant will join the Department for Outcomes at World Bank Group headquarters in Washington, D.C. S/he will be embedded in the AI for Outcomes workstream within the Outcomes Delivery Unit, focusing specifically on expanding and deepening the work on the quality assurance of the department's suite of AI tools and analytical applications. S/he will report to Sereen Juma, Manager, Outcomes Department (WKCOD). This position requires a combination of strong analytical and technical skills, hands-on experience with AI/ML systems and large language models (LLMs), a rigorous approach to testing and evaluation, and the ability to communicate quality and risk issues clearly to both technical and non-technical stakeholders. The role offers the opportunity to work across World Bank Group practice groups and on cross-cutting priorities in a fast-paced environment. The AI Quality Assurance E T Consultant will be responsible for:

Advertisement
  • Designing and executing QA frameworks for the department's suite of AI tools, including LLM-based ap-plications for outcomes measurement and analysis. This includes developing structured test plans, evaluation protocols, and benchmark datasets to systematically assess tool performance, accuracy, and reliability.
  • Conducting structured testing and evaluation of AI outputs — including LLM-generated text, data ex-tractions, classifications, and analytical summaries — against defined quality standards, flagging inconsistencies’, hallucinations, or outputs that deviate from expected behavior.
  • Supporting model selection and version change evaluations, including conducting comparative assessments when underlying LLM versions are updated, deprecated, or replaced, and maintaining version control records that enable performance changes to be correctly attributed over time,
  • Apply Natural Language Processing (NLP) evaluation techniques, such as semantic similarity analysis and text embedding models, to systematically assess the consistency and quality of AI-generated text out-puts across testing cycles.
  • Developing and maintaining QA documentation, including test case registries, issue logs, version-specific evaluation reports, and traceable records of changes made in response to identified quality gaps.
  • Collaborating with data scientists and engineers on the AI for Outcomes team to translate QA findings into actionable technical improvements, supporting iterative refinement of models, prompts, and retrieval pipelines.
  • Monitoring and tracking usage of AI tools across the Outcomes Delivery Unit's portfolio of applications, including maintaining dashboards or logs that capture user adoption, query volumes, feature utilization, and error rates. Synthesizing usage data into regular reports to inform prioritization decisions, identify under-performing tools, and surface patterns that may indicate emerging quality or user experience issues.
  • Preparing clear and well-structured QA reports and briefings that communicate findings, risk assessments, and recommendations to technical teams and non-technical stakeholders, including department management.
  • Supporting additional work program priorities of the Department for Outcomes as needed, including contributions to knowledge products, tool demonstrations, stakeholder engagement, and outcome data calculations.
{{waiting}}
Este puesto ya no está abierto.
Antes de postular

Revisamos la calidad de los anuncios, pero una organización puede modificar una vacante en cualquier momento: confirme el puesto y la fecha de cierre en su sitio oficial antes de postular. Ningún empleador legítimo cobra por postular, por una entrevista ni por tramitar una candidatura — cualquier petición de pago es una estafa. Si algún dato parece incorrecto o el anuncio parece fraudulento, informe sobre esta vacante.

Advertisement