AI Literacy for High School Students: Why Systems Thinking and Critical Thinking Matter (Part 2)

What are the risks, unintended consequences and responsibilities that come with AI use?

Image courtesy of Canva

In Part 1 of our series ‘AI Literacy for High School Students: Why Systems Thinking and Critical Thinking Matter’, we explored the promise of AI: its ability to support learning, accelerate research, improve decision-making and open new possibilities across sectors. But AI’s benefits do not exist in isolation.

Every AI system is shaped by data, design choices, incentives, access and power, and these same conditions can also produce risks, unintended consequences and uneven impacts.

In this second part, we turn to the other side of the system: what AI can disrupt, who it may exclude, and what responsibilities come with using it.

The risks: cheating, mistrust and misinformation

The same capabilities that make AI useful also create problems, and one obvious concern within the education sector is cheating, an issue that arises with the widespread use of generative AI. If students can ask a tool to produce an essay, solve a problem set or write a reflection, then assessment design must change. But the deeper issue is not only whether a student used AI - it is whether learning happened. This is where systems thinking helps. A narrow response says, “Ban AI.” A systems response asks, “What does our assessment system reward? Are we assessing final products only, or also process, reasoning, drafts, critique and application? Are students using AI because they are lazy, or because the task feels disconnected from meaningful learning? Do staff have the time and training to redesign assessment?”

Another consequence is social mistrust. People are now hyper-aware of certain writing styles: well-written, but somehow flat and impersonal. The irony is that many of these styles existed long before generative AI, as we need to note that AI trains on existing databases. Corporate writing, consultancy language and academic abstraction have often sounded stoic and rigid, set within a certain formula or tone. But AI has started making people suspicious of language itself. A student who writes clearly may be accused of using AI. A colleague who uses em dashes or balanced paragraphs may be suspected. A second-language English speaker who uses AI to improve grammar may be judged differently from a native speaker who uses an editor.

Attempts to navigate growing mistrust around writing styles have produced the strange market for “humanising” AI writing. Within this market, many generative AI companies now coin and promote the idea of “Human-Centred AI”. The very phrase reveals the problem. If a machine generates text and another machine rewrites it to appear less machine-generated, what exactly is being humanised? Style? Error? Voice? Plausible imperfection? In a trust-based system, writing is not only a product. It is evidence of thought, relationship, accountability and context. When that link weakens, people may begin to distrust not only AI-generated writing but all writing.

Misinformation is an even larger concern. Large language models generate text by predicting likely sequences of words based on patterns in training data. However, these systems may produce plausible-sounding but inaccurate text, often called hallucinations, and that this is especially problematic when people rely too heavily on outputs without proper consideration of risk. In a world where AI can generate convincing articles, images, audio and video, the cost of producing misinformation falls dramatically. The burden of verification shifts onto readers, teachers, journalists, employers and citizens. Another risk is bias, as AI systems are shaped by the data they are trained on, the assumptions of the people who design them, and the contexts of deployment. If data is incomplete, skewed, outdated or drawn mainly from dominant languages and contexts, the outputs can reproduce those limitations. Chapman University Hanna et al., 2025 NIST, 2022

This can have much deeper consequences very quickly. When people cannot easily distinguish authentic from synthetic, trust in institutions, media, evidence and expertise can erode. The problem is not only that false things may be believed. It is also that true things may become easier to dismiss. “That could be AI-generated” can become a convenient way to avoid accountability or mislead people.

Image courtesy of Canva

The deeper consequences: resource and environmental costs, agency and exclusion

The environmental costs of AI are often hidden behind clean interfaces. Large AI systems depend on data centres, chips, cooling systems, electricity grids and global supply chains. There are significant concerns around the environmental costs of training and running large AI models. One estimate suggests that by 2026, data centres could use close to 1,050 TWh of energy, placing them, if counted as a country, among the world’s largest energy consumers – ahead of Japan and just behind Russia. Because much of the world’s electricity still comes from fossil fuels, and because intermittent renewables such as solar and wind can’t provide the stable, high density power that data centres require, AI remains heavily reliant on carbon based energy sources, This gives AI a significant significant carbon footprint, especially as generating new content tends to be more energy and carbon intensive than classification tasks, with image-related tasks generally requiring more energy than text-only tasks. In response to the high carbon intensity through energy use, several major AI and technology companies including Meta, Microsoft and Amazon are exploring nuclear energy as a stable, resilient and lower emissions energy source, although this presents wider concerns around safety, waste, regulation and geopolitical tensions.

Water use is also part of AI’s environmental footprint because the data centres that power these systems produce substantial heat and often need cooling systems to keep servers running safely. In some cases, this cooling depends on water, which can place added pressure on local water resources. Another environmental concern is the significant electronic waste, when spent components are discarded. They can release toxic chemicals like lead or mercury into the environment.

On one hand, AI models or systems may help improve the efficiency of energy, water, transport and agriculture systems, but its expansion also increases demand for computing power. This creates a systems paradox: the same technology that may support environmental solutions can also contribute to environmental strain. The key question, then, is not simply whether AI is good or bad for the environment, but which AI is being used, by whom, for what purpose, at what scale, with what energy source, and with what level of transparency.

Another deeper consequence is loss of agency. If people increasingly outsource thinking, writing, planning, remembering and deciding to AI systems, they may gradually weaken the very skills they need to use those systems wisely. Critical thinking is not automatic – it is practised. So are judgement, attention, interpretation, ethical reasoning and creativity. AI can support these skills, but it can also dull them if used passively. Multiple emerging studies suggest that increased reliance on AI, particularly regenerative AI and large language models may affect cognitive skills. One study suggests that overreliance on technologies such as AI could reduce critical thinking skills by up to 75%, while another study on brain activity during AI-assisted tasks found significantly lower neural engagement compared with tasks completed independently. This suggests that AI can enable cognitive offloading and greater efficiency, but it may also weaken opportunities for sustained mental effort like higher-order thinking, creativity and problem-solving. Kosymna et al., recently found that cognitive engagement decreased as external assistance increased: participants working without tools showed the strongest neural connectivity, search-engine users showed moderate engagement, and LLM users showed the weakest. This is especially important in education and professional development. A learner who uses AI to challenge their assumptions, compare arguments and improve a draft may become stronger. University of Oxford Nature A learner who uses AI to bypass reading, reflection and struggle may become dependent. The difference depends partly on how learning is designed, partly on personal discipline, and partly on the wider system surrounding the learner.

AI is also not inclusively designed by default. Many tools are developed in wealthy countries, trained largely on dominant languages and internet-accessible data, and optimised for users with stable connectivity, digital confidence and the ability to pay. This unevenness is shown in data by Microsoft showing that the majority of the Global South constitutes less than 20% of AI User Share, compared with the Global North which is at least twice that level of adoption, suggesting far greater AI diffusion across wealthier economies. Large language models also struggle to process linguistic differences and have limited ability to understand contexts, which matters for students from the Global South, multilingual learners, people working in low-resource settings, and communities whose knowledge is underrepresented online.

For these learners, AI literacy cannot mean simply adopting the latest tool. It must include understanding whose data is included, whose language is prioritised, whose knowledge is missing, and whose realities are misread. A student working on climate adaptation in a rural community, for example, may find that an AI tool gives confident but generic advice based on assumptions from very different contexts. Without critical awareness, AI can reproduce a hierarchy in which local knowledge is treated as less valid than machine-generated generalisation.

Towards responsible and ethical AI use

Despite the concerns and negative aspects, we cannot avoid AI altogether, because at this stage in its development, that is neither realistic nor fair. AI is already embedded in search engines, office software, phones, learning platforms, recruitment systems, financial services and public debate. The better question is how we create opportunities to learn AI tools responsibly and ethically. This is where the Rondil Scholars Research Program comes in, by teaching practical use alongside critical thinking and understanding.

In our 12-week online research program for high school students, learners gain opportunities to experiment with prompts, compare tools, evaluate outputs, check evidence, identify hallucinations, protect sensitive information, understand bias, and decide when not to use AI. As part of our research and career development program for secondary school students, scholars also have space to discuss authorship, consent, intellectual property, labour, environmental costs and inequality.

By employing a systems-thinking approach, we give scholars a better framework than hype or panic. It allows them to see AI as both useful and risky, empowering and extractive, efficient and costly, creative and standardising. This approach reflects the type of future-ready skills for students increasingly valued by universities and employers. It encourages learners to look for feedback loops, unintended consequences and uneven impacts. AI can accelerate work, research and innovation, support learning, discovery and access. But it can also intensify mistrust, misinformation, dependency, exclusion and environmental strain. The task ahead is not simply to become better AI users, but to become better stewards of the systems AI is changing. Through AI literacy education, critical thinking for students, and interdisciplinary research opportunities, young people can develop the knowledge and judgement needed to navigate an increasingly AI-driven world.

At Rondil Scholars, we believe students should not only learn how to use emerging technologies but also understand their wider social, economic, and ethical implications. Our online research program for pre-university students helps learners explore complex global challenges, develop independent research skills, and make more informed decisions about their academic and career pathways.

Written by Dr Sandile Mtetwa-Omuthe, Dr Manuel Kernen, Dr Ronald-Omuthe