Situated Evidence: Policymaking evidence in existing African language AI evaluations

1. Introduction

Foundation models do not perform equally well across languages, and in most African languages, their performance lags behind English. A model that gives accurate health information in English may give less accurate information in Hausa. Safeguards that hold when a harmful request is made in English may also fail when the same request is made in isiZulu. Recent research finds that safety benchmarks built in English overestimate how safe models behave in African languages (Abdullahi et al. 2026). Yet for the millions of people who will use AI systems in African languages, a model's performance in their language is crucial, because it determines whether the system is reliable and safe for them.

At the same time, many argue that African policymakers cannot easily ground AI policy in local evidence. They argue that foundation models are built elsewhere and that existing evaluations are largely designed and validated in Western settings, in English and other widely spoken languages (Ireri et al. 2026). Disclosure has also declined. The 2025 Foundation Model Transparency Index found that average disclosure by major developers fell from 58 to 40 out of 100 in a single year, with training data among the least disclosed areas (Wan et al. 2025). One of the most important pieces of information for African users, how these systems behave in their languages and settings, is often the information developers are least likely to provide. Governments therefore must rely on assessments produced by developers or foreign institutions, which provide little information about how these models behave for African users.

Evaluations measuring model performance in African languages begin to address this problem. To assess model performance in African languages, researchers have built benchmarks and evaluation datasets that test tasks such as translation, question answering, reasoning, and safety. They often design these resources with native speakers and domain experts (Nekoto et al. 2020; Abdullahi et al. 2026). Some evaluations now even test performance in specific sectors such as AfriMed-QA, which evaluates medical question answering using questions from more than 60 medical schools in 16 African countries (Nimo et al. 2025). This article argues that, when measured carefully, models' performance on these evaluations is early policy evidence. They already show where systems fall short, in which domains and for which communities. They also give African policymakers a locally grounded basis for deciding what oversight of these systems should take into account.

This article begins by introducing the African-led evaluations that now exist and explains why language evaluations are its main focus. It then sets out what these evaluations already indicate about model performance, what they cannot yet establish, which policy proposals are beginning to draw on them, and which questions they raise for locally grounded oversight.

2. Introducing African evaluations of models

Over the years, African researchers and their collaborators have built a substantial body of evaluations testing how foundation models perform in African languages and, increasingly, in African contexts. Much of this work has come from African research communities such as Masakhane, which pioneered participatory methods that involve native speakers directly in building datasets and benchmarks (Nekoto et al. 2020). Recent resources go further. For example, UbuntuGuard, a safety benchmark, starts with harmful queries written by 155 domain experts, from which it derives locally grounded safety policies (Abdullahi et al. 2026).

The philosopher Donna Haraway's notion of "situated knowledges" helps explain why this emerging body of work is important. Haraway argued that all knowledge is produced from a particular location, and that claims to a neutral "view from nowhere" usually conceal one (Haraway 1988). Evaluations conducted mainly in English are therefore situated in the languages, contexts and assumptions of the places where they were designed. African-language evaluations, by contrast, test systems in the languages African users speak, and several are designed with experts from the continent. This makes them a more direct guide to how those systems affect African users.

What African-Language Evaluations Measure

African-language evaluation has grown from a handful of datasets into a broad set of benchmarks, largely led by African research communities. The evaluation resources most relevant to policy fall into three groups:

1. evaluations measuring general language capability,

2. evaluations measuring knowledge and accuracy in consequential domains, and

3. evaluations measuring safety.

 

As the table shows, the evaluations were built differently. IrokoBench and Uhura are translations of English tests. AfroBench mixes translated tests with tests first written in African languages. AfriMed-QA and UbuntuGuard start from questions written by African experts and institutions, though UbuntuGuard's were written in English and then machine-translated. Because of these differences, results from one evaluation cannot always be compared directly with another, as Section 5 explains.

Most of the evidence also still concerns general capability. Evaluations in consequential domains and of safety are newer and fewer, but they are the ones most directly relevant to oversight. The AI Safety Institute's joint testing is notable because it involved governments, and Kenya took part in the international exercise, which tested whether safeguards held across languages (International Network for Advanced AI Measurement, Evaluation and Science 2025a). Kenya's AI Safety Institute later contributed Kiswahili test material to a third joint exercise focused on AI agents and harms such as fraud and leaks of sensitive information (International Network for Advanced AI Measurement, Evaluation and Science 2025b).

Beyond language: Other African-rooted evaluations

Language is not the only dimension along which African contexts test AI systems. A growing set of evaluations examines how models perform on African knowledge, values and real-world tasks:

• Health - AfriMed-QA tests medical knowledge using questions from African medical schools in 16 countries (Nimo et al. 2025), and a real-world study at Penda Health clinics in Nairobi examined how an AI tool affected clinical errors across nearly 40,000 patient visits (Korom et al. 2025).

• Agriculture - The AIEP Initiative evaluated AI advisory services for smallholder farmers in Kenya, including user surveys and curated agricultural question sets (Collis et al. 2025).

• Bias and stereotypes - AfriStereo, built from stereotypes collected in Senegal, Kenya and Nigeria, found statistically significant stereotypical bias in nine of eleven models tested (Le Beux et al. 2025).

• Cultural knowledge - BLEnD, covering 16 countries and regions including Ethiopia and Nigeria, found that models perform markedly better for cultures well represented online (Myung et al. 2024).

• Reasoning and education - NSMQ Riddles, drawn from Ghana's National Science and Maths Quiz, found that leading models performed worse than the best student contestants (Boateng et al. 2026).

• Finance and safety - SafeAlert tests models against Nigerian fraud scenarios (Uduimoh et al. 2026), and the GSMA and Zindi African Trust and Safety LLM Benchmark holds more than 4,000 verified adversarial tests contributed by the Zindi community, with Swahili, Hausa, Yoruba, Igbo, Zulu, Afrikaans, Amharic and Akan among the languages most represented (GSMA 2026).

Some evaluations already combine domains. Uhura tests health, legal and financial knowledge in African languages, and UbuntuGuard tests locally grounded safety policies in ten African languages.

3. Language evaluations as the dominant form of evaluation and their importance

This article uses language evaluations as its main example, for three reasons.

1. First, they form the largest and most mature body of African-rooted evidence: they are mostly peer-reviewed, cover dozens of languages, and their findings recur across independent studies. Evaluations in other domains are valuable but fewer, and several are recent preprints or industry-partnered studies.

2. Second, language cuts across every other domain. A health, agricultural or public-service tool intended for African communities must first work in the languages those communities use.

3. Third, language evaluation most directly illustrates what situated evidence offers, because it tests systems under the conditions African users actually encounter. A fuller review of African-rooted evaluations across domains would be a valuable complement as that evidence matures.

Why language evaluation matters for development-focused AI

Much of the case for AI in Africa is made in development terms. The African Union's Continental AI Strategy calls for "an Africa-centric, development-focused approach to AI" and links AI to the aspirations of Agenda 2063 and the Sustainable Development Goals (African Union 2024). The most commonly proposed deployments reflect this, such as advisory services for smallholder farmers, support for health workers and patients, learning tools for students, and access to public services.

These deployments are meant to serve people who are often least well served by systems built and tested mainly in English. They include rural communities, smallholder farmers, primary-care patients, and learners, many of whom communicate primarily in African languages and access digital services through voice, messaging, or intermediaries such as extension officers and community health workers. With only 36% of Africa's population using the internet in 2025 (ITU 2025), many of those development-focused AI aims to reach are not yet online and will likely encounter AI through local-language, voice-based interfaces when they do.

Experience in education shows why the language of delivery matters for outcomes. UNESCO's Global Education Monitoring Report estimates that as much as 40% of the world's population does not have access to education in a language they speak or understand, and cites evidence from Côte d'Ivoire that in 2008, 55% of grade 5 pupils who spoke the language of the test at home had mastered basic reading, compared with 25% of those who did not (UNESCO 2016). There is good reason to expect a similar relationship for AI-mediated services, where information may be technically available but poorly rendered in the user's language and therefore may not deliver its intended benefit.

Early deployment experience points the same way. An initiative supported by the Gates Foundation and GIZ that deployed AI advisory services for farmers in Kenya and India reported high user satisfaction, but identified weak speech recognition and translation for low-resource languages as a persistent obstacle to voice access (Collis et al. 2025). In Nairobi, a real-world study of an AI tool supporting clinicians at Penda Health found 16% fewer diagnostic errors and 13% fewer treatment errors among clinicians with access to it (Korom et al. 2025). That study shows what AI can contribute in a clinical setting, but it was conducted in English and used trained clinicians. Community-facing tools that speak directly to patients in their own languages would raise different questions.

Language is also becoming a stated priority in national AI policy. The 2026 Global Index on Responsible AI, which assessed 135 countries, found that 47 countries have frameworks addressing cultural and linguistic diversity in AI, and 52 countries have government-led initiatives, many of them focused on local-language technologies, multilingual datasets and national or regional language models. Kenya's National AI Strategy 2025–2030, for example, highlights natural language processing as a tool for promoting the country's linguistic diversity, including chatbots that work in local languages. However, the Index also found that few governments require developers to adapt their systems to local contexts (Adams et al. 2026). Evaluating how models perform in African languages offers one way to check whether these commitments are met in practice.

The implication is that deployments with the greatest development ambition often target the communities for whom evidence about language performance matters most, and where that evidence is thinnest. Most existing evaluations are also text-based, while many intended users will interact by voice. If AI in African languages works well, it could considerably widen access to information and services. Language evaluation helps policymakers and implementers understand whether, and for whom, it does.

Why language evaluation matters for governance beyond data protection

Evidence on language performance also broadens the questions African AI governance can ask. Data protection law has so far been the main legal instrument through which African states have engaged with AI, and by the end of 2025, at least 44 African countries had enacted such laws (Buza 2026). The African Union's Continental AI Strategy treats these frameworks as foundational. However, analysts note that data protection alone cannot address risks such as algorithmic bias, discrimination and loss of cultural identity (Akana and King'ori 2024). Whether a chatbot answers a health question accurately in Hausa is primarily a question of reliability and safety, which data protection law addresses only indirectly. Researchers have also argued that African policy responses to AI should reflect the continent's cultural and linguistic diversity rather than adopting external models wholesale (Gwagwa et al. 2020). Evidence about how models perform in African languages gives policymakers a concrete basis for doing so.

4. Policy signals from the evidence

Thus far, the evaluations point to five findings, each raising a consideration for policy and oversight:

a. Performance in English is not a reliable guide to performance in African languages

The most consistent finding is a substantial gap between English and African languages. AfroBench reported large gaps on most of its tasks (Ojo et al. 2025). IrokoBench, using human-translated test sets in 17 languages, finds a significant gap between high-resource languages such as English and French and the African languages tested, and finds that the best open model reaches about 63% of GPT-4o's performance. Results improve when test items are first translated into English, which suggests that part of the weakness lies in handling the African language itself (Adelani et al. 2025).

Consideration: performance claims based on English-language benchmarks may not describe the experience of people using a system in an African language. This affects how public bodies assess systems they procure and what information users receive about a system's limitations.

b. Evidence about one African language does not carry over to another

Performance varies widely among African languages and tracks the amount of training data available (Ojo et al. 2025). The authors of the Sahara benchmark argue that these data differences are partly "policy-induced", the legacy of historical choices that favoured colonial languages (Adebara et al. 2025). A 2025 review found that only 42 African languages receive any support in the models it examined, with four languages accounting for most of it (Hussen et al. 2025).

Consideration: evidence is useful for oversight only if it covers the languages people actually use. The finding also suggests that data and language policy, alongside technology policy, shape how well AI serves different communities.

c. Gaps extend into consequential domains

Uhura tests scientific knowledge and truthfulness in six African languages, with questions spanning health, law, finance and politics, and finds that all models performed better in English (Bayes et al. 2026). AfriMed-QA tested 30 models on medical questions drawn from African institutions and found significant variation across specialities and geographies. Its human evaluation also found that consumers consistently preferred model-generated answers to clinicians' answers (Nimo et al. 2025), a reminder that confident answers may be trusted even when accuracy is uneven.

Consideration: the significance of a performance gap depends on the use. A shortfall that is a minor inconvenience in casual conversation may matter considerably in health information, financial advice or public services. This points to focusing attention on high-stakes uses rather than AI in general.

d. Safety behaviour does not transfer automatically across languages

Translating harmful requests into low-resource languages, including Zulu, bypassed GPT-4's safeguards with a 79 per cent success rate (Yong et al. 2023). Tests across EU languages found higher jailbreak success in lower-resource languages for some frontier models (Kanepajs et al. 2024). UbuntuGuard found that English-centric safety benchmarks overestimate safety in African languages (Abdullahi et al. 2026), and the AI Safety Institutes' joint exercise found that safety behaviour varied across the ten languages it tested (International Network for Advanced AI Measurement, Evaluation and Science 2025a). The picture also changes over time. A 2026 replication found that newer models had largely closed attacks using a single translated message. However, attacks that spread across longer conversations in low-resource languages, including Swahili, still produced harmful responses 42 to 71 per cent of the time, depending on the model and language (Marx and Dunaiski 2026, as reported in Onuoha et al. 2026).

Consideration: safety testing conducted by developers in English may not reflect how systems behave when used in African languages. Because translation tools are freely available, users who do not speak a language can exploit weaker safeguards in that language. Closing one weakness also does not prove a system is safe in a language, because the gap can shift from one attack type to another.

e. Local norms can be tested even when models are built elsewhere

UbuntuGuard's design is as significant as its results. It starts from harmful queries written by 155 domain experts. From these, the authors derived safety policies grounded in local contexts and tested guardian models against them. Guardian models are the safety filters that screen what a chatbot receives and produces. A separate proposal for Nigerian fintech similarly evaluates models against Nigerian fraud scenarios and banking communications (Uduimoh et al. 2026).

Consideration: this speaks directly to the problem of governing systems developed abroad. African states are beginning to assert jurisdiction over foreign providers. Kenya's draft AI policy, for example, would extend to foreign providers whose systems have direct and foreseeable effects in Kenya (Issaias and Khaoma 2026). Enforcing such obligations is difficult in practice, as South Africa's Competition Commission has acknowledged regarding technology companies operating from other jurisdictions (Panchia 2025). Testing how a system behaves against locally specified expectations does not resolve that question of enforcement reach. It does, however, work without access to a system's training data or internal design, and so offers a way to ground oversight in local conditions even when developers disclose very little.

5. The limits of evaluations

Evaluations are the main tool available for understanding how AI systems behave, but the evaluation research community has itself documented significant limitations. Most benchmarks were built to compare and improve models, and their limits matter when people use them for policy. Some limitations apply to evaluation in general; others are specific to African-language evidence.

General limitations

Construct validity: A benchmark is useful only if it measures what it claims to measure. A review of 445 language model benchmarks found that about half measured abstract concepts such as reasoning without clear definitions, and that only 16 per cent of studies used statistical tests when comparing models (Bean et al. 2025). Researchers have also warned more broadly against treating narrow benchmarks as measures of general capability (Raji et al. 2021).

Data contamination: Models trained on web-scale data may have encountered benchmark test items during training, inflating their scores and making them appear more capable than they are (Sainz et al. 2023). The risk grows as benchmarks become widely used and their contents circulate online.

Sensitivity to test conditions: Small changes in question formatting can substantially change results. One study found performance differences of up to 76 accuracy points from formatting changes alone in one open model (Sclar et al. 2024). A single reported score may therefore reflect the chosen test set-up as much as the model.

Capability and impact differ: Weidinger and colleagues distinguish three layers of evaluation: a system's capabilities, how people interact with it, and its wider effects on society. They find that existing evaluations concentrate on the first and argue that context determines whether a capability causes harm (Weidinger et al. 2023).

Results date quickly: Scholars of technology regulation have long described a "pacing problem", in which science and technology advance faster than the legal and ethical mechanisms meant to oversee them (Marchant et al. 2011). General-purpose AI models sharpen this problem. New model versions appear within months, capabilities shift between releases, and evaluation results can quickly go out of date. One study documented GPT-4's accuracy on a task falling from 84 to 51 per cent between its March and June 2023 releases (Chen et al. 2024).

Uneven coverage of concerns: Public benchmarks cover some risks far better than others. An analysis of nearly 195,000 benchmark questions concluded that they are insufficient on their own for assessing compliance with the EU AI Act (Prandi et al. 2025).

Automated grading: Many evaluations now use language models to grade other models' outputs. The AI Safety Institutes' multilingual exercise found differences in reliability between automated and human evaluators (International Network for Advanced AI Measurement, Evaluation and Science 2025a).

Limitations specific to African-language evidence

Comparability across languages: A gap between two languages is meaningful only if the test items are equivalent. Translated items help, but they can carry the original language's cultural assumptions; the Global MMLU study found that model rankings change when culturally sensitive items are analysed separately (Singh et al. 2025).

Translation and verification quality: Many African-language evaluations are built by translating English material, and the quality of that translation is not always well checked. An audit of AI safety datasets found that UbuntuGuard's Hausa policy material scored 66.37 on the authors' own translation-quality measure, below their acceptance threshold of 70. Swahili material produced by the same process scored above 93 (Onuoha et al. 2026). The same audit found that a dataset claiming coverage of many languages can contain only a few hundred machine-translated items for a single African language, checked by one person.

Uneven language coverage: Few African languages appear in most evaluations, while most of the continent's roughly 2,000 languages are not evaluated at all (Hussen et al. 2025). Coverage also varies by harm type. The audit found no natively written safety data on self-harm or sexual content for either Hausa or Swahili, and no self-harm data of any kind for Hausa. One evaluation in a language also does not mean that language has been fully assessed. Kenya's Kiswahili contribution to the international agentic testing exercise addressed fraud and leaks of sensitive information, but not other safety risks such as harmful content or jailbreaks (Onuoha et al. 2026).

Grading in low-resource languages: Where automated graders are used, they may perform worse on the African languages being tested, adding another source of error. In the agentic testing exercise, Kiswahili was one of three languages where automated grades differed most from human ones (Onuoha et al. 2026).

Models and deployed systems differ: Most benchmarks test a model in isolation, while services built on it add retrieval, fine-tuning, filters and human review, which can raise or lower performance. Few, if any, published evaluations examine deployed systems with African-language users in specific services, where decisions about particular uses would ultimately be made.

While these limitations do not remove the value of the evidence, they call for careful interpretation. For example, we are not yet at the point of considering benchmark scores as verdicts, as that would overstate what they can show. Consistent findings across many independent evaluations, using different methods, languages and models, are more robust than any single benchmark result, and the African-language findings described above recur across studies. The evidence is therefore strong enough to identify where performance and safety concerns are likely, and which questions deserve attention.

6. Policy proposals already drawing from the evaluations

Researchers and policymakers have begun to develop practical proposals based on this evidence. They differ in scope and ambition, and several remain contested. The table below summarises the main ones:

 

Two patterns stand out. Most proposals concern the production and disclosure of evidence (testing, reporting, shared evaluation frameworks, better data) more than restrictions on use. African proposals also tend to emphasise local definition, including institutions, sectors and countries, specifying what they expect of systems and testing against those expectations.

Comparative precedent also exists outside Africa. The EU AI Act requires instructions for high-risk systems to state, "when appropriate, its performance regarding specific persons or groups of persons on which the system is intended to be used" (European Union 2024). Language-specific performance is one natural application of that principle, though the EU has not specified it as such.

7. Questions for locally grounded oversight

While the evaluation evidence does not prescribe a particular response, it suggests questions policymakers, regulators, and public institutions may want to consider as they develop their approaches.

• Which languages and uses matter most? Evidence is useful only if it matches the languages people actually use and the purposes for which they use a system. Identifying priority languages and high-stakes uses is a natural first step, and avoids treating all AI uses alike.

• What evidence should accompany systems used in high-stakes settings, and who should produce it? Developers, deployers, public testing bodies and independent researchers each have different access, incentives and capacity. The scarcity of deployment-level evidence suggests that public research alone will not fill the gap.

• Can existing tools carry some of this work? Public procurement, data protection impact assessments and sector supervision already provide occasions to ask how systems perform for local users. For example, most data protection laws require impact assessments to account for the "nature, scope, context and purposes" of high-risk processing. Using such tools may be possible without new legislation.

• How can oversight avoid discouraging African-language services? Many communities are underserved by existing services in their languages. Demanding requirements could push providers to offer services only in well-resourced languages, and they would weigh most heavily on smaller and African developers building for African languages. Any response would need to be weighed against the realistic alternative: not providing the service at all.

• Where would regional cooperation help? Kiswahili, Hausa, and many other languages cross borders. Shared test sets, common documentation and mutual recognition of evaluations could spread costs and give smaller states access to evidence they could not produce alone. Shared testing could also ease the difficulty noted above of acting on systems whose providers sit outside any single jurisdiction. Researchers have also argued that sharing evaluation methods and findings would let resource-constrained institutions extend what each can test on its own (Ireri et al. 2026).

• How can oversight keep pace? Given how quickly models change, a one-off evaluation has limited value. Periodic re-testing and monitoring after significant updates may matter more than any single assessment. This is the practical response to the pacing problem discussed in Section 5.

8. Conclusion

African policymakers face real constraints in governing AI systems that change quickly, are built elsewhere and are documented only partially by their developers. They do not, however, lack homegrown evidence. A body of African-language evaluation, built largely by African researchers, already shows that English performance poorly predicts performance in African languages, that gaps vary by language, that they reach into health, law, and finance, and that safety behaviour does not reliably carry across languages. It also shows that systems can be tested against locally defined expectations without access to how they were built. These questions of reliability, safety, and equitable service extend well beyond data protection, and the evaluations provide a basis for asking them.

None of this establishes what form oversight should take, and the evidence has clear limits, above all the scarcity of evaluations of real deployments. It does provide a locally grounded starting point: an indication of which questions matter for African users and where attention is most warranted. As policy and oversight frameworks develop across the continent, this evidence helps ensure they respond to how AI systems actually perform for the people who use them.

References

Abdullahi, Tassallah, Macton Mgonzo, Mardiyyah Oduwole, et al. (2026), UbuntuGuard: A Culturally-Grounded Policy Benchmark for Equitable AI Safety in African Languages, in: Findings of the Association for Computational Linguistics: ACL 2026, 33262–33276.

Adams, Rachel, Fola Adeleke, Ayantola Alayande, et al. (2026), Global Index on Responsible AI: 2026 Report, 2nd edition, Global Center on AI Governance.

Adebara, Ife, Hawau Olamide Toyin, Nahom Tesfu Ghebremichael, et al. (2025), Where Are We? Evaluating LLM Performance on African Languages, in: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 32704–32731.

Adelani, David Ifeoluwa, Jessica Ojo, Israel Abebe Azime, et al. (2025), IrokoBench: A New Benchmark for African Languages in the Age of Large Language Models, in: Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), 2732–2757.

African Union (2024), African Union Committed to Developing AI Capabilities in Africa, press release, 28 August, African Union Commission.

Akana, Chuma, and Mercy King'ori (2024), The African Union's Continental AI Strategy: Data Protection and Governance Laws Set to Play a Key Role in AI Regulation, Washington, DC: Future of Privacy Forum.

Bayes, Edward Thomas, Israel Abebe Azime, Jesujoba Alabi, et al. (2026), Uhura: A Benchmark for Evaluating Scientific Question Answering and Truthfulness in Low-Resource African Languages, in: Proceedings of the Fifteenth Language Resources and Evaluation Conference (LREC 2026), 1485–1504.

Bean, Andrew M., Ryan Othniel Kearns, Angelika Romanou, et al. (2025), Measuring What Matters: Construct Validity in Large Language Model Benchmarks, in: Advances in Neural Information Processing Systems 38 (NeurIPS 2025), Datasets and Benchmarks Track.

Boateng, George, Naafi Ibrahim, Samuel John, et al. (2026), NSMQ Riddles: A Benchmark of Scientific and Mathematical Riddles for Quizzing Large Language Models, arXiv preprint 2605.07051.

Buza, Maria (2026), The Year of the Teeth: Data Protection in Africa Roundup, 2025, Projections for 2026, St. Gallen: Digital Policy Alert (first published by Tech Hive Advisory, 31 December 2025).

Chen, Lingjiao, Matei Zaharia, and James Zou (2024), How Is ChatGPT's Behavior Changing Over Time?, in: Harvard Data Science Review, 6, 2.

Collis, Stewart, Florence Kinyua, Vikram Kumar, et al. (2025), Building AI-Based Advisory Services for Smallholder Farmers: Technical Learnings from the AIEP Initiative, arXiv preprint 2601.11537.

European Union (2024), Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 Laying Down Harmonised Rules on Artificial Intelligence (Artificial Intelligence Act), Official Journal of the European Union, L series, 12 July.

GSMA (2026), African Trust & Safety LLM Benchmark: Stress-Testing AI Safety Across Africa's Languages and Contexts, GSMA Newsroom, 29 July.

Gwagwa, Arthur, Erika Kraemer-Mbula, Nagla Rizk, et al. (2020), Artificial Intelligence (AI) Deployments in Africa: Benefits, Challenges and Policy Dimensions, in: The African Journal of Information and Communication, 26, 1–28.

Haraway, Donna (1988), Situated Knowledges: The Science Question in Feminism and the Privilege of Partial Perspective, in: Feminist Studies, 14, 3, 575–599.

Hussen, Kedir Yassin, Walelign Tewabe Sewunetie, Abinew Ali Ayele, et al. (2025), The State of Large Language Models for African Languages: Progress and Challenges, arXiv preprint 2506.02280.

International Network for Advanced AI Measurement, Evaluation and Science (formerly the International Network of AI Safety Institutes) (2025a), Multilingual Joint Testing Exercise: Improving Methodologies for LLM Evaluations Across Global Languages, Singapore AI Safety Institute, 26 June.

International Network for Advanced AI Measurement, Evaluation and Science (2025b), International Joint Testing Exercise: Agentic Testing – Advancing Methodologies for Agentic Evaluations Across Domains: Leakage of Sensitive Information, Fraud and Cybersecurity Threats, Singapore AI Safety Institute and UK AI Security Institute, 17 July.

Ireri, Gathoni, Cecil Abungu, Jean Cheptumo, et al. (2026), Assessing the Case for Africa-Centric AI Safety Evaluations, arXiv preprint 2602.13757.

Issaias, Ariana, and Taria Khaoma (2026), Kenya: AI Governance Framework Continues to Take Shape – Draft Artificial Intelligence and Emerging Technologies Policy, 2026, Bowmans, 31 July.

Issaias, Ariana, Mumbi Kiragu, and Sahara Ali (2026), Kenya: Artificial Intelligence Bill, 2026 – Proposed Too Soon?, Bowmans, 23 April.

ITU (International Telecommunication Union) (2025), Measuring Digital Development: Facts and Figures 2025 – Internet Use, Geneva: ITU.

Kanepajs, Artūrs, Vladimir Ivanov, and Richard Moulange (2024), Towards Safe Multilingual Frontier AI, arXiv preprint 2409.13708.

Kenya (2019), Data Protection Act, No. 24 of 2019, Kenya Gazette Supplement No. 181, Nairobi: Government Printer.

Korom, Robert, Sarah Kiptinness, Najib Adan, et al. (2025), AI-Based Clinical Decision Support for Primary Care: A Real-World Study, arXiv preprint 2507.16947.

Le Beux, Yann, Oluchi Audu, Oche D. Ankeli, et al. (2025), AfriStereo: A Culturally Grounded Dataset for Evaluating Stereotypical Bias in Large Language Models, NeurIPS 2025 Workshop on Centering Low-Resource Languages and Cultures in the Age of Large Language Models, arXiv preprint 2511.22016.

Marchant, Gary E., Braden R. Allenby, and Joseph R. Herkert (eds) (2011), The Growing Gap Between Emerging Technologies and Legal-Ethical Oversight: The Pacing Problem, The International Library of Ethics, Law and Technology, 7, Dordrecht: Springer.

Marx, Dylan, and Marcel Dunaiski (2026), Multilingual Jailbreaking of LLMs Using Low-Resource Languages, arXiv preprint 2605.18239.

Myung, Junho, Nayeon Lee, Yi Zhou, et al. (2024), BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languages, in: Advances in Neural Information Processing Systems 37, Datasets and Benchmarks Track, 78104–78146.

Nekoto, Wilhelmina, Vukosi Marivate, Tshinondiwa Matsila, et al. (2020), Participatory Research for Low-resourced Machine Translation: A Case Study in African Languages, in: Findings of the Association for Computational Linguistics: EMNLP 2020, 2144–2160.

Ngugi, Kevin (2026), Kenya AI Policy Draft Released for Public Consultation, Techweez, 28 July.

Nimo, Charles, Tobi Olatunji, Abraham Toluwase Owodunni, et al. (2025), AfriMed-QA: A Pan-African, Multi-Specialty, Medical Question-Answering Benchmark Dataset, in: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 1948–1973.

Ojo, Jessica, Odunayo Ogundepo, Akintunde Oladipo, et al. (2025), AfroBench: How Good Are Large Language Models on African Languages?, in: Findings of the Association for Computational Linguistics: ACL 2025, 19048–19095.

Onuoha, Chialuka Prisca-Mary, Bright Etornam Sunu, and Rashidat Sikiru (2026), Language-Specific Gaps in AI Safety Training Datasets, arXiv preprint 2608.13695.

Panchia, Yeshiel (2025), Competition Commission Warns: AI and Big Tech's Grip on Media Threatens South Africa's Democratic Foundations, Daily Maverick, 24 February.

Peppin, Aidan, Julia Kreutzer, Alice Schoenauer Sebag, et al. (2025), The Multilingual Divide and Its Impact on Global AI Safety, arXiv preprint 2505.21344.

Prandi, Matteo, Vincenzo Suriani, Federico Pierucci, et al. (2025), Bench-2-CoP: Can We Trust Benchmarking for EU AI Compliance?, arXiv preprint 2508.05464v2.

Raji, Deborah, Emily Denton, Emily M. Bender, et al. (2021), AI and the Everything in the Whole Wide World Benchmark, in: Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, 1.

Sainz, Oscar, Jon Campos, Iker García-Ferrero, et al. (2023), NLP Evaluation in Trouble: On the Need to Measure LLM Data Contamination for Each Benchmark, in: Findings of the Association for Computational Linguistics: EMNLP 2023, 10776–10787.

Sclar, Melanie, Yejin Choi, Yulia Tsvetkov, et al. (2024), Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I Learned to Start Worrying about Prompt Formatting, in: Proceedings of the Twelfth International Conference on Learning Representations (ICLR 2024).

Singh, Shivalika, Angelika Romanou, Clémentine Fourrier, et al. (2025), Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation, in: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 18761–18799.

Uduimoh, Andrew Anogie, Hadiza Umar Yusuf, and Oluwafemi Osho (2026), Context-Aware Pre-Deployment Evaluation of AI Systems: A Regulatory Framework for Nigerian Fintech, arXiv preprint 2609.24016.

UNESCO (2016), If You Don't Understand, How Can You Learn?, Global Education Monitoring Report Policy Paper 24, Paris: UNESCO.

Wan, Alexander, Kevin Klyman, Sayash Kapoor, et al. (2025), The 2025 Foundation Model Transparency Index, arXiv preprint 2512.10169.

Weidinger, Laura, Maribeth Rauh, Nahema Marchal, et al. (2023), Sociotechnical Safety Evaluation of Generative AI Systems, arXiv preprint 2310.11986.

Yong, Zheng-Xin, Cristina Menghini, and Stephen H. Bach (2023), Low-Resource Languages Jailbreak GPT-4, NeurIPS 2023 Workshop on Socially Responsible Language Modelling Research.