Why you need to use less AI-powered learning tools
When I started building AI-powered tools for education, my instincts were the same as most people’s. That meant sending the user’s input to the AI, getting the response, and displaying it. AI was that tool. Everything ran through it.
It worked fine until it didn’t. The API key expired during a sabbatical, and no one noticed for weeks. The service went down from time to time, and the tools went down with it. And at some point, you realize you’re paying a monthly fee for an AI API that ends up answering the same 12 questions over and over again. This started to feel more like a dependency I built for myself than a solution.
I’ve created about six learning tools over the past year, and around the third or fourth one, the approach I wish I had understood from the beginning started to take shape. That means AI should be the finishing layer, not the foundation.
In this article…
Principle: Do the work before calling the API
Every AI API call has a cost measured in tokens sent and received. However, the monetary costs are actually the most obvious. What took me a while to understand are latency costs (API calls add a few seconds of user experience that you don’t get with local lookups), reliability costs (external dependencies become points of failure that you can’t control), and environment costs. Environmental costs are easy to ignore at the scale of a single query, but they accumulate across tools serving hundreds of users per day.
The approach I arrived at is simple and straightforward. The idea is to leverage structured data and deterministic logic as much as possible before AI intervenes. If you can answer the question with keywords that match your curated knowledge base, do so. If you can use a user dropdown selection to filter the set of recommendations to the relevant ones, do so. Save the AI API calls for the parts of the task that actually require generation functionality. This part requires you to personalize or synthesize your response in a way that pre-written answers cannot provide.
To be clear, this is not an argument against the use of AI. This is an argument for using it correctly.
what actually happens
The FAQ chatbot I built for my organization’s help desk is probably the most intuitive example. The version most people would build first, and the version I mostly built, takes all the user questions, sends them along with the entire FAQ database to an AI, and lets the model decide which answers to return. This works, but it means that every question including “What time do you work?” costs a token, adds a few seconds of latency, and requires an active connection to an external service.
What I ended up building uses a hybrid approach. When a user enters a question, the tool first performs keyword matching against a curated set of FAQ entries. Each entry has keywords and synonyms associated with it, and matching logic calculates a confidence score. If the confidence level is high enough, the tool will immediately return a pre-written answer. No AI API calls are required.
The tool only escalates to AI if the keyword match doesn’t yield reliable results. Even then, the entire knowledge base will not be dumped to the prompt. Only the top candidate entries, those with the highest scores but below the confidence threshold, are sent along with the user’s original question. AI gets a focused set of possibilities, not all of them.
What I like about this pattern is how widely transmitted it is. Whether the tool is an internal support bot, a coaching recommendation engine, or a resource finder integrated with your LMS, the architecture is the same. In other words, process what you can with your structured data, and send only what you can’t resolve to your generative model.
Less context often yields better answers
This is the part that surprised me. We expected to save money by sending less data to the AI. We did not expect such a noticeably positive response.
In the behavior intervention tool I built for educators, users choose grade bands, behavior categories, frameworks, and settings. The tool uses these selections to filter the research base for strategies before the AI recognizes them. Instead of sending 200 strategies and asking the model to choose the one that is relevant, send 15-20 strategies that already match the user’s context and ask the model to integrate them and make personalized recommendations.
This changes the job of AI from search and filtering to synthesis and personalization. The prompts are small and the materials are already relevant, so the model doesn’t spend processing power sifting through the noise. As a result, the output is more focused. I’ve seen this pattern hold true across multiple tools, and consistently see pre-filtering as a quality improvement, not just a cost reduction.
The broader point applies to anyone building or evaluating tools that send knowledge bases to AI. That is, can that knowledge base be filtered using a deterministic method first? Dropdown selection, keyword matching, category filters, role-based scoping. Structured data that narrows the fields before the generation call does double duty, saving cost and removing the noise that the model has to navigate.
What if AI is not available?
This is the question that separates tools built for demos from those built for continued use, and I’ve learned to ask this first rather than last. The tool I built works hierarchically. Once a valid API key is present and the call is successful, the user gets a complete AI-enhanced experience. If the key is not found or the call fails, the tool falls back and returns the curated content directly. Users continue to get filtered and relevant resources that match their input. They just don’t have an AI-generated personalization layer on top. I turn on this fallback mode by default for tools I hand over to other organizations. Because the circumstances in which AI will disappear are predictable. API keys expire, billing lapses over the summer, or new administrators don’t realize that the tool relies on external services.
This is something I think is overlooked in the conversation around AI-powered learning tools. In most cases, the real value comes from carefully selected content from known sources. A tool that returns “Here are 4 research-based strategies that match your selection” along with an explanation and source information gives educators and L&D professionals something they can act on and test. AI adds conversational sophistication and situational specificity, but the content is AI-independent. If a tool doesn’t function at all without an active AI connection, that vulnerability can present itself at the worst possible time.
A quick check before building (or buying)
Whether you’re building a learning tool in-house or evaluating one from a vendor, a few questions can help uncover how much of the tool’s value actually relies on AI.
Will core functionality work without AI?
This is the most obvious question because it forces a distinction between tools that are essentially AI products and tools that use AI to enhance something that already works. These two have very different risk profiles. What data can be filtered or structured before an AI call?
Every drop-down selection, keyword match, or category filter that narrows down the context before the generation call creates a double burden of cost and quality. Is there a response I can pre-write instead of generating?
For common questions and standard recommendations, human-generated answers stored in a database are faster, cheaper, and often more reliable. Generating something new each time is solving a problem that doesn’t exist. What if the API is not available?
The answer should not be “the tool stops working.” Does the data need to leave the user’s environment?
Local processing should be the default, especially for anything related to sensitive information, such as education or human resources. Even simple, imperfect local checks (such as regular expression pattern matching on names and ID numbers) keep the data within your organization’s control, rather than being routed through an external API to determine whether the data is sensitive. The irony of sending potentially private data to an external service to see if it contains private data is one of those design problems that only becomes apparent later.
Why this matters more than cost savings
While the discussion here is not primarily about cost savings with AI API credits, for organizations with limited budgets, this is a real consideration. A deeper question is what kind of technology we choose to rely on.
Tools that require constant connectivity to external AI services to function are vulnerable exactly where it matters most when an organization’s landscape changes. And in every organization I’ve ever worked with, things change regularly. Budgets change, staff changes, and priorities change. Tools that gracefully degrade as the AI layer disappears are tools that will survive the transition. The darkened items are tools that will be replaced.
I keep coming back to the question that underlies all of this. If we believe that AI should help us learn rather than the other way around, what does that actually look like at the level of tool architecture?To me, it started to look like a tool that uses AI as effectively as possible while minimizing reliance on it. The goal is not to avoid AI. The goal is to use it when it really adds something, and handle everything else in a way that is simpler and that the team can actually see and control.
