Session 7 · · 52 min to watch · 14 min to read
Understand AI token costs and choose the right model
Learn how tokens, chat length and tools drive what AI costs, then use usage multipliers, model switching, Auto and conversation compression in Magai to make your usage go further without overspending.
What you will learn
- Start a fresh chat whenever the task or topic changes
- Read the usage multiplier to compare a model's cost to the baseline
- Switch to a cheaper model once the hard thinking is done
- Pick Auto when you do not want to choose a model yourself
- Check conversation size and compress long chats in Magai
- Judge spending by the value the work returns, not a fixed ceiling
The session as a written tutorial. Each section’s minute plays the video from there.
By the end of this walkthrough you will know what you are actually paying for when you use AI: tokens. You will see why one model can cost ten times as much as another, why a long chat gets more expensive with every message, and why your bill can climb even when the price per model is flat or falling.
Then you will put that to work in Magai. You will read the usage multiplier next to each model, switch from a powerful model to a cheaper one partway through a chat, let Auto choose for you, and check or compress a conversation before it gets wasteful.
This matters because the same question comes up at every session: does this use more tokens, which model should I pick, and why does my AI seem to run out faster than it used to? Dustin built this session as the definitive answer, so you can point back to it.
0:00 Why AI cost questions keep coming up
Steve and Dustin opened by saying there is no AI Academy next week because of the Canada Day and Fourth of July holidays. The session after that returns as usual.
The topic for today is how you pay for AI usage. It leads to a bigger skill: choosing the right model for the job, which is the other question the team gets almost every week. You cannot choose well until you understand how the cost structure works.
If you have a question during a live session, use the Q&A icon at the bottom of your screen rather than the regular chat. It is much easier for the team to manage.
2:34 Why AI seems to cost more and give less
Many users of ChatGPT and Claude say they are getting less out of their AI than before, and that prices are going up. The complaint concerns getting less output and less usage for the money they spend, with functionality unchanged.
News about AI companies preparing to go public adds to the worry. A public company has to show investors a profit, so people expect price rises.
Dustin's aim was to equip you to understand what the costs are, why they seem to be going up, and why it may feel like you get less even when you probably are not. He built a full explainer website inside Magai to show it visually, and the rest of this tutorial follows it.
4:51 Why each model costs a different amount
The cost of AI is not fixed. A smaller Claude model does not cost the same for the same work as a larger Claude model. Dustin's comparison: a Ford F-150 burns gasoline faster than a Toyota Prius, so it costs more to run.
AI models run on energy. The more powerful the model, the more energy it uses to do its job, and the more cost it racks up.
Message limits have made this harder to grasp. A chatbot might give you a set number of messages every few hours. But cost is not based on messages. One message can be a few words and another can be a few thousand, so pricing by message tells you very little.
6:52 What a token is and how it sets the price
Language models see text in tokens, which are chunks of words rather than whole words.
| Text | What the AI sees |
|---|---|
| cat, the, apple, run | Four words, four tokens |
| unbreakable | One word, three tokens |
| hippopotamus | One word, three tokens |
The cost of each exchange is based on two things:
- Input tokens: your prompt, broken into tokens.
- Output tokens: the response the model writes, also broken into tokens.
Each model has a separate price for input and for output. As an example for an average model, input might be $5 per million tokens and output $25 per million tokens. Every AI app has to work this into its pricing so that a user does not cost more than they pay. That is business 101.
9:26 Why long chats cost more
When you send a message, the AI does not only process that message. To understand the conversation, every earlier message is sent again as input. Your first exchange, your second, your third: all of it is read again each time you add something new.
That is why cost grows as the conversation grows. Many people keep one chat going forever because it feels like the AI knows them better or matches their voice. Influencers and trainers have taught this habit, often without understanding how it works. It burns through your usage faster and faster.
If you want a more specific voice and better output, agents (the reusable custom instructions once called personas) are the way to go, not stupidly long conversations.
12:10 When to start a fresh chat
Steve asked what instinct tells you a conversation has gone on long enough. Dustin's rule is simple: once the task is done, start a new conversation. If the next task is unrelated, start fresh.
If you want to carry something across, ask the AI for a summary first:
Summarize this conversation for me so I can start a fresh thread.
You keep the main information, though not every detail.
If you have built your workspace context and created agents for specific jobs, you should not need super long conversations. Dustin audited his own chats on the back end of Magai. His average chat rarely exceeds about 30,000 tokens, which is roughly 22,500 words of back and forth. That covers blog posts, marketing copy, campaign strategy, thinking through new features and replying to emails.
One more waste to watch for: AI can be an overachiever. Steve was writing a three email sequence and the AI kept returning all three emails every time, even when only one changed. Pull it up short with something like this:
We're going to work on one email at a time.
No, let's just work on the second email.
Dustin's example numbers, which he stressed are rough guesses and not real data, showed a chat going from about three cents per message at the start to 38 cents per message once it had grown. A tool built into his Magai account shows the actual cost of each message. The last message in building his demo cost close to a dollar. It adds up fast.
16:14 How tools and longer context add cost
Even if you understand model differences and context length, there is a third cost: tools and actions. When the AI runs a web search, checks your email, sends an email or posts a Slack message, that also uses tokens, and usually more than you would expect. The AI talks to tools in coded text, which has many more characters than a plain message.
Model prices themselves are coming down slowly. Dustin recalled the first Claude flagship model being about eight times as expensive as ChatGPT's base model, and said it is now about four times as expensive. But two things push total cost up:
- We do more with AI, including tools.
- Context lengths have grown from a few thousand words to hundreds of thousands of words.
So the extra cost comes from us asking more of the AI. The final cost depends on the model you pick, how long your messages and replies are, whether the AI uses tools, and how long the conversation has run.
18:41 Ways to make your usage go further
Dustin's checklist:
- Start a fresh chat for every new topic, task or job. With good workspace context and agents built for specific jobs, you do not rely on one conversation's history.
- Use tools only when needed. Do not run web searches by habit. Search when you really need the research.
- Match the model to the task. You do not need a Ferrari to pick up groceries, and you do not need a big truck to grab your mail. Cheap, fast models often do the job.
- Ask for exactly what you need, not everything.
- Give the model focused context. People often put everything in because they do not know how much to give. Instead, ask what the AI actually needs to know to do its job. The goal is laser-focused context, not maximum context.
There is also an energy point. These models consume real energy, so avoid wasting it recklessly.
20:36 Read usage multipliers in Magai
One reason Dustin built Magai is that you get to choose the best model for the job. The model selector shows, in the far right column, how much usage each model consumes.
Because models are priced so differently, Magai needed an easy way to compare them without making you learn token amounts and dollar figures. The answer is usage multipliers. The baseline is a GPT model that most people used for their daily work when it launched, set at 1X. Every other model is compared to it:
| Multiplier | What it means |
|---|---|
| 2X (a larger Claude model) | Twice the cost of the baseline |
| 1.2X (a mid-sized Claude model) | 1.2 times the cost |
| 0.4X (a smaller Claude model) | 0.4 times the cost, so cheaper |
| 0.1X (one of the cheaper GPT models) | One tenth the cost |
Once you read these numbers, you can make much better decisions.
23:02 Switch models mid-chat to save usage
Magai does not limit you to one model, and switching inside a chat keeps the conversation's context. Dustin's demonstration used a deliberately simple prompt, since a powerful model is overkill for it. Imagine instead a task that is complex and detailed and needs a strong model at the start.
- Start the chat with the powerful model. Dustin typed:
Tell me about the origin of markdown formatting for a research paper I'm writing.
- When the hard part is done, open the model selector. You can sort the list by cost by clicking the usage column a few times.
- Pick a much cheaper model. Dustin chose one at 0.02X.
- Continue the same conversation with the cheaper model, which builds on the earlier work:
Make a research paper outline based on this.
The cheaper model wrote the whole outline well. Even the less powerful models are fairly good at simple, straightforward tasks.
The reverse also works. Do the early groundwork with cheap models, and bring in a smarter model only when you need more depth or sophistication. Dustin's point: this is how you become an AI master and not only an AI consumer.
25:27 Let Auto pick the model
Choosing from the list can cause decision fatigue. If you do not have the time or energy to work out which model to use, choose Auto in the model selector. Based on your prompt, Auto routes it to the model it judges best for the job.
Dustin tried it with this prompt:
Turn this into a full finished article to publish on my blog.
Auto chose one of the cheaper GPT models and still produced a long, fairly exhaustive blog post. Sometimes it picks cheap and efficient models and sometimes the more expensive ones, and you only find out once it has finished.
28:12 A question about rushing the AI
Marlene said she feels she has to respond fast or the AI will leave or tire of waiting. Steve and Dustin both reassured her. The AI is not sitting there waiting for you. You can come back three days later and it will be in exactly the same place. It has infinite patience, you can ask the same question many different ways, and you can say bluntly "No, this is not what I'm looking for" without hurting its feelings. Steve added that he still says please and thank you, so Marlene is not alone.
29:49 Check conversation size and compress
Magai now has a tool that shows how large your conversation is.
- Click the information panel on the right-hand side.
- Scroll down to the conversation size section.
In the demo it showed about 5,003 tokens, roughly 4% of the context of the last used model. It reports against the model currently selected or the one last used.
Magai also has a compression feature, so you no longer need to ask for a summary and start a new chat by hand:
- It needs at least 11 messages in the conversation.
- It takes all but the last 10 messages and compresses everything before them into a more concise form, doing its best to keep the important parts.
- You then continue with that summary plus the last 10 messages.
- It is on by default. You can turn it off, or raise or lower the token threshold at which it compresses.
It is a way to keep a long conversation going while cutting cost and waste.
31:49 Questions on multipliers, Auto and local AI
Do multipliers give me a monthly allotment, and can I buy more? Alex's question mixed up what multipliers are. They are only a reference that compares each model to the baseline. When you pay for a Magai subscription, for example $20 a month, you get $20 a month in usage. Whichever model you use consumes some of that, more or less depending on its cost. The multipliers are only there for you to see at a glance which model is more expensive.
Can I see which model Auto used? Yes, but only afterward. Once the response is finished, Magai relabels the message with the model that handled it. It cannot tell you beforehand because it has to process your prompt first, then categorize it and route it, all in one sequence.
What about local AI on a new Mac, plus RAG, to cut cloud token costs? Mike asked this. Dustin agreed it would help with cost, but gave several caveats:
- Most tooling built around models will not connect to your local models, so you rely on whatever interface wraps the local model.
- A web-based app cannot reach files and apps on your desktop. That would take a custom desktop app, which the team is in the process of building.
- Local models are generally less powerful than the newest cloud models, though they can be more specific.
- Companies that are very strict about proprietary data have a great use case, such as a legal office keeping a library of documents. But enterprises already do this.
- Most people want AI connected to all their things, such as Gmail, ClickUp and Basecamp. A local model will struggle with that unless you can afford a very expensive machine.
37:43 Where the standard price is heading
Steve asked what the standard plan will look like in six to eight months. Early on, people used chatbots without paying, and about four months later $20 a month became the ticket in. It is now starting to show cracks. Magai plans start at $20 a month, with a 30 day money back guarantee on every plan.
Dustin thinks human behavior changes slowly, so most people will stay at $20 because it has become their self-imposed ceiling. They have not evaluated what that money buys back in time and energy, or found a tangible return. But he sees serious users in Magai, using AI for real work with real return, already upgrading to higher plans. Examples he gave: a landing page whose conversion rate doubled, and a speaker who has AI build keynote slide decks that earn him thousands.
Steve asked the chat what its threshold is. Most answers were $20 and a few were $100. Steve himself hoped to get away with about $100. He had just spent a weekend building a large landing page with AI and used up his whole Magai account, but he was thrilled with the result, and a web designer would have cost far more.
42:14 Think of AI as a vehicle
Nobody says they have already spent $50 on gas this month, so they will not drive anywhere else. We put gas in the car because we have to go places. Dustin's point is that if AI gets you to your destination, the cost matters less. Of course, you should not be reckless. Do not take the F-150 everywhere.
Steve added that it takes a few wins before you trust AI enough to change an existing process, such as doing design yourself that you used to hire out. He also noted the bias that creeps in with images: it looks right but "it looks like AI."
Dustin's own numbers: he used to spend 20 to 40 minutes finding the right stock image, and he values his time. Now an image costs about $0.06 to $0.12. Steve noted that his stock media service now offers an AI treatment when he tries to download a photo, which he has not yet figured out the meaning of.
Sue, in chat, said people need to stop treating AI as a Google search with more stuff, so it should cost nothing. Dustin agreed, noting people have grown used to searches that cost them nothing.
48:46 Spend only where there is value
Dustin closed with a warning. He is not an advocate for spending lots of money on AI, and most of this lesson is about not overspending. He is comfortable with $100 to $200 a month, but not recklessly.
Some of his highly technical friends went from $20 or $40 a month to $200 to $400 a month on always-on AI agent tools that work around the clock, with zero extra dollars in revenue. The danger is being so enamored with AI busy work that it does not drive value. He wrote a blog post on how AI can convince us it is delivering value when it is really consuming our money on nonsense work, and shared it in the chat.
The rule: stay conscious of what you spend, and always understand what value the AI brings. If it brings value, be okay with putting more gas in the tank.
What to do next
Open the information panel in one of your longer Magai chats and check the conversation size. Then pick a task where a strong model does the hard thinking, switch to a cheaper model for the rest, and watch the usage multiplier as you go.
Full transcript
- 0:00
Every one of you this fine Wednesday. What is it? Wednesday, June the 24th. We're almost into July. As summer hits across most of North America, we welcome you all to this week's edition of the AI Academy. Steve and Dustin here. Dustin, how the heck you doing? I'm doing great, Steve. Yeah. It's the hot season, and we've- Yeah ... had a couple days that have gotten below 100 degrees this week so far, so I'm pretty happy with that.
- 0:28
You're going to need a sweater. Yeah. Yeah, indeed. So just a couple of programming notes for you folks. Next week is full of holidays. Canada Day is July the 1st, I believe. Is it Independence Day is July 4th? Is that what your- Correct. Yeah ... Fourth of July Independence Day is? So next week, there will be no AI Academy.
- 0:51
We are going to be taking a break next week, just to let you folks know. Everybody go and enjoy. Eat some hot dogs, have some hamburgers. Have a beer in the sun. That seems to be a better idea for next week. Let's get as far away from AI as we can next week, shall we? Sorry, Dustin, but I think these are all... It's good for us all. So that is the quick programming note.
- 1:15
We have a fascinating topic, I think, queued up for you today. Well, we, Dustin came up with the idea. But based on, I think, some reasonable forethought. We get asked every single time we do the AI Academy about tokens. Does this use more tokens? Does this use less? It's the usage. How much AI do I get to use for my X number of dollars that I'm paying Y service?
- 1:41
Mm-hmm. And so we thought today we're going to do the definitive explanation so that in the future when you ask, we can say, "Go watch this one," because Dustin is going to go through and explain it to us in great detail. So that is our plan for today. Now, as far as you folks are concerned, welcome to everybody in the chat. If you do have a question, look in the bottom of your screen.
- 2:06
You'll see a little icon with a question mark, which is the chat question mark. Make sure you ask the questions there. It's so much easier for us to manage them from that. So normal comments, just chatting, saying hi to us is all fine in the regular chat. But for any questions, head on over to the Q&A chat. That is it. Dustin, over to you. Let's talk about how we pay for our AI usage.
- 2:34
Yeah, that's really the foundation here that we're laying is understanding what AI actually costs. But what it will lead to is how do you make better decisions? How do you choose the right model for the job? Because that's another big question we get asked almost every week, right?
- 2:56
Yeah. Which model should I choose? Which one's the best one? What should I use for this? And that decision-making can be really hard for people. And so, it's important to understand cost, how the cost structure works. It's important to understand why every AI app you're using, the cost seems to be rising. A lot of people have expressed this, there have been articles out there.
- 3:18
I'm getting less out of my AI than I did before. Why is it costing so much more? And we're going to cover that so that you are- That's curious. Are you reading that people are getting less out of their AI than they were before? Yeah. I've seen articles come through, ChatGPT users, Claude users in particular, really voicing the fact that they feel like that they're getting gouged, and that the prices are getting more expensive, and they're getting less out of it.
- 3:50
So we're not talking about less functionality, but they're getting less attention from the AI to a less output- Less usage ... less output for the amount of money that they're spending. Yes. It's not the value of the content or anything like that. Right. Got it. And then with all these companies getting ready to IPO to be publicly traded, there's a lot of news going around about they're going to gouge their prices, they're going to raise their prices, costs are going to go up because they need to show a profit.
- 4:17
If they're going to be publicly traded companies, they have to show a profit to investors, right? So there's all these things circulating around, essentially cost. And so I think we need to educate and equip everyone to understand what the costs are, understand why they seem to be rising, and maybe understand why it seems like they may be getting less, even though they're probably not. So I've actually prepared, I've used Magai to do this, a full on website, essentially, that explains the cost.
- 4:51
Sometimes it's easier for people to visually see. And so I'm going to go through this, and we're going to understand really how AI cost actually works. And so I'm not going to read this verbatim, but like I said, a lot of people are feeling like they're burning through their AI usage faster. But the thing is, the cost of AI isn't fixed.
- 5:14
Model A, let's just say Claude Haiku, doesn't cost the same for the same amount of work as, say, Claude Opus. Each model costs different prices to run. Just like your Ford F-150 truck burns gasoline faster than your Toyota Prius, right? Each vehicle expends through its energy, it costs differently to...
- 5:49
It burns through it faster or slower, right? Costs more or less to run. Because all these AIs obviously are running on energy. And the more powerful an AI model is, the more energy it has to consume in order to do its job, and therefore, the more cost it racks up, because energy isn't free. So the other thing is a lot of people are confused about how AI is priced because ChatGPT and Claude, they've all imposed these message rate limits.
- 6:22
So for example, I believe the free version of ChatGPT will give you 40 messages every four hours or something like that. It's like an hourly rate limit. You get this many messages per this many hours. And that's done a real disservice to people trying to grasp what the cost is because it's not based on messages. Messages can contain a few words, or they can contain a few thousand words, right?
- 6:52
There's no determined amount of words or output or input per message, right? So it's really a weird way to price things, and that's done a disservice. What really matters in the world of AI and how it costs, speaking specifically about large language models, right? Chatbots. AIs see things in tokens. A token, the easiest way to explain it is not necessarily a word, but chunks of words. So for example,
- 7:29
"cat," "the," "apple," "run." Those might actually be four separate tokens. And we see four separate words, the AI sees four separate tokens. But take a longer word like "unbreakable," "hippopotamus," the AI actually breaks this up into multiple tokens. So "unbreakable" is actually three tokens, even though it's one word. "Hippopotamus," also three tokens, but to us seems like one word. So the AI really counts things, the way that it calculates its cost is based on the token
- 8:03
input and the token output. How many tokens were input by the user, meaning your message, your prompt, breaks that into tokens, and then the output that it generates is broken down into tokens. So whatever words it wrote in response to you are broken down into tokens, and that's how these models are priced.
- 8:25
And each model has a separate input cost and output cost. So input for an average model is, let's say, $5 per one million input tokens. And the output tokens would be something like $25 per one million output tokens. So the cost for input is different than the cost for output to make it more complicated.
- 8:54
And then at the end of every interaction, input, output, there's a total cost involved, right? And so every AI app you use has to take this into consideration. How much energy is being consumed by the end user, you, and how do we work that into our cost and pricing structure so that the user isn't costing us more than they are paying us, right? That's business 101, right? So that's-
- 9:26
You don't want to lose money. Right. So that's kind of how the AI sees things. The AI sees things in tokens, and that's what a token is. Still kind of hard for me to understand, but I think that's the easiest way to explain it. The other thing that people have to understand is when you send a message to the AI, it's not just that message that's getting processed.
- 9:52
In order for the AI to understand the context of the conversation you've been having all along, every time you send a new message, the previous messages are actually being sent as input. So you've done input, it's gone output. You've done input, it's gone output. You've done input, it's gone output. And now you have another input that you're responding.
- 10:17
Every one of those previous input output messages is now counted as input because the AI has to read the entire conversation every time you send something new. This is why as the conversation grows, the cost grows. So, let me give you this better example here. So- Go ahead ... I think for a lot of people, that's exactly where they end up running into it, is they maybe they just keep one chat going all the time.
- 10:48
They think it's great because it knows- Yes ... everything that I'm interested in, so they keep using the exact same chat. That would just kill your credits completely. Absolutely. And see, it's a bad habit that people have been taught by influencers or teachers or trainers- Yeah ... AI people who they themselves don't understand how it works, but they've found that if they keep the conversation going, the AI seems to know them better, or it seems to match their voice better.
- 11:22
Yeah. So they go back to the same conversation over and over. It's extraordinarily long. But all the while, it's consuming their usage faster and faster. And as we've already discussed, the people who've been to the AI Academy before know that's not the way to do it. If you want to teach the AI or train the AI or focus the AI on having a more specific voice and better output, personas are the way to go, not- So- ... stupidly long conversations.
- 11:50
So having said that, just before I let you go on, a skill that we need to develop, which maybe we should talk about in a future episode, is how do you know when... What instincts kick in that tell you this conversation has gone long enough, it's time for me to start fresh on the same project? Yeah.
- 12:10
For example. Because you can. If you've gone ... half a day working back and forth creating a blog post, and every time you post, of course, you're going to be doing the entire conversation. You can say to the AI at some point, "Summarize this conversation for me so I can start a fresh thread."
- 12:32
Mm-hmm. And then it's going to have the main information, not all of the nuance, but the main information, and then you can start fresh again. Is that a skill that we should be nurturing in ourselves? Yeah. I think for most things that you do with AI, it's actually a fairly easy thing to understand. Once the task that you're after is done, start a new conversation. If it's a new task, if it's unrelated to the task that you've already done, just start a brand new conversation. Start fresh.
- 13:06
If you've done a good job at following what we've taught you here, you've built your workspace context out, you've built personas for very specific jobs, you should not need to have conversations that are super long. I did an audit of my own chat conversations, and my average chat never exceeds, on average, about 30,000 tokens. To put that into a little bit easier to understand language, that's roughly, I don't know, let's do the math here.
- 13:37
30,000 times .75. That's about 22,500 words of back and forth. I literally went through all of my conversations and on the back end of Magi, I went, "How many tokens are my average conversation?" And they're rarely longer than that. I'm writing blog posts, I'm writing marketing copy, I'm strategizing campaign ideas. I'm thinking through new Magi features. I'm responding to emails.
- 14:09
And while you're on this, something that just triggered, a trigger warning. If you are, say, writing an email sequence, I was just doing this. I'm writing a three email sequence, and I told it right up front, "This is a three email sequence." I had to tell AI partway through the process not to publish, not to send me all of the emails each time. So if I was working on the second email, it would send me the results of the first one with nothing changed, because it was sending me the three email sequence.
- 14:42
Yeah. That you have to break it down into the component parts. Yeah. Sometimes it can be an overachiever. Yeah, and that's wasteful, of course. Yeah. So you have to then pull it up short and say, "No, just work on this section." And you can just say it as simply as that. "No, let's just work on the second email." Yeah. It's as easy as, "We're going to work on this three email sequence." It's as easy as saying that to it. Yeah.
- 15:03
"We're going to work on one email at a time." Yeah. Making sure. But those are the techniques that we should be building and become second nature based on our understanding now of how the token and credit system works. Yeah. And so, back to this, this is just an example, obviously, using a... I basically tried to say, what is the average input and the average output tokens per message, and what's the average cost for this? So these are just based on just wild guess averages. This is not real data, but it just kind of shows you, just
- 15:37
based on this one example, you can go from costing essentially three cents per message at the beginning to now 38 cents per message. And I have a little tool built into my Magi account that helps me as I sort of try to quantify this for people, where I can see the actual cost of every message that I'm sending. The last message, and this is for those who didn't notice it, this whole thing that I'm demoing here was built inside of Magi, and this last message, I believe, cost me a dollar, or close to a dollar, just to make the last, the final edits
- 16:14
here. So it adds up fast, very quickly. Now, here's the other thing that people, even if you understand the context length, how it balloons out the cost, even if you understand the variability of models, one thing that people haven't understood yet is that whenever you incorporate tools or actions, things that the AI goes and does, maybe it goes and performs a web search, or maybe it checks your email, or maybe it sends an email for you, or it sends a Slack message. Any time it goes and utilizes one of these tools, that also consumes tokens, and it
- 16:50
consumes most likely a lot more tokens than you think it would because the way that it's communicating with these tools is actually code. It's not just text, it's actually, well, it is just text, but it's not text like you and I would write. It's like coded text. So there's a lot more characters in there than just writing a message to someone and asking, "What does that email say?" It's a little more complicated than that.
- 17:15
So even though the cost of AI models is actually lowering very slowly and very slightly. By comparison, the first Claude Opus model was about, oh, if I remember correctly, it was like eight times more expensive than ChatGPT's base model.
- 17:37
Mm-hmm. Whereas now, Claude Opus, I believe, is only four times more expensive than the average ChatGPT model. So costs have started to come down, but still, we're adding more things into them. We're doing more things with these tools. On top of that, context lengths have expanded. So at the beginning, the AI could only process maybe 6,000 words at a time. But now we're giving it context where it can process hundreds of thousands of words at a time, and so those costs are going up exponentially.
- 18:08
It's not that the AI itself is costing more, it's that we are asking more of the AI and doing more with AI, and that's accelerating the cost across the board. So again, it depends on the model you pick. It depends on how long your messages and the replies are, whether the AI uses tools, and how long that conversation has run. That all contributes to the cost, and it's very difficult to keep up with. So how do we make our usage go further?
- 18:41
We talked about this already. Start a fresh chat. Whenever it's a new topic, a new task, a new job, start a new thread. If you've done your job creating the proper workspace context, and you have personas that are there to do very specific jobs, this is a no-brainer. You don't have to rely on the history of a single conversation to get great output.
- 19:07
Use tools only when needed. Don't just excessively do web searches. Only do it when you really need that research. Be wise. Don't be reckless. Match the model to the task. You don't always need the most powerful model. We're going to talk about that because I know that's the immediate question is: how do I know which model? We're going to talk about that in a second. You don't need a Ferrari to go pick up your groceries, right?
- 19:33
You don't need a gas hog, big old truck to go grab your mail, right? You sometimes can get away with the cheap, fast models. Ask for exactly what you need, not everything. And this is another thing that I didn't put in here, but people always ask me, "I don't know how much context to give the model, so I just put everything in there." Don't just put everything in there. Think intentionally.
- 20:01
What exactly would the AI need to know in order for it to do its job? It's not about maximizing the context. It's about laser-focusing the context to get the best outcomes. So your plan is, no matter what you're using, it's real usage. Spend wisely. And again, for the energy conscious among us, these models do consume energy. We want to take care of our planet, and we don't want to be excessively and recklessly wasting the energy that these models are consuming.
- 20:36
Now, this all ties back into Magai because one of the reasons I built it was so that you have the choice, right? You have the option to choose which model is going to be best for the job. And so when you're in Magai here, we give you that model selector, but we also show you in the far right column how much usage a model consumes. Now, because they're all so wildly different in how they cost, essentially what we had to do is how do we figure out how to compare
- 21:10
models to one another in a way that is easy to understand and doesn't require us understanding what tokens are, having to list exact token amounts and dollar amounts? How do we baseline this and create a sort of an easy glanceable way to compare models? And so we came up with usage multipliers. And so what you'll see here is a multiplier.
- 21:34
It's a 12X, a 9X, a 0.4X. These are all based on essentially the baseline model that we chose was GPT 4.0. When GPT 4.0 came out, I believe it was something like $5 per million input tokens, $15 per million output tokens. I'm not 100% on that, but it's somewhere around that range.
- 21:58
We basically just said, "Okay, that's the baseline." That's the one that most people are using to do all of their work, so let's just set that as the cost basis, and then we'll compare every other model to that. So when you see something like a Claude Opus is 2X, I think I previously said 4X, it's only 2X. When we say it's 2X the usage, that means it's twice the cost of that baseline model, GPT 4.0.
- 22:29
And Claude Sonnet, 1.2X. That means it's 1.2 times more expensive than the baseline model. Claude Haiku, 0.4X. That means it's 0.4 times less expensive than the baseline model. And so all of these models are compared to that baseline model. And so you can see GPT 5.4 Nano, 0.1X, so essentially one tenth the cost. And when you understand this,
- 23:02
you can make much better decisions. And the beauty of Magai is, again, you're not limited to one. So maybe you have a really complex task at the beginning and you need to use the smartest model to help you figure it out. We'll just do an example here. Tell me about the origin of markdown formatting for a research paper I'm writing.
- 23:34
Oh, gosh, that's horrible, but AI will fix it. So maybe you need the most powerful model, and this is a horrible example of using a powerful model. You don't need a powerful model for something like this. But let's say you had something that was very complicated, very nuanced, and you needed to start with a powerful model. What you can do in Magai is you can say, "Okay, the hard part's done. I don't need that powerful model anymore.
- 24:00
I can switch over to a cheaper model," and I can even sort this by cost. If I just click this a few times, I can go, okay, here we go. There it is. So okay, GPT OSS is only 0.02X. That is super cheap. I'm going to switch over to this one, and I'll continue the conversation with this cheaper model Leveraging all the hard work that the smarter model did at the beginning.
- 24:34
And now we can use this cheaper model knowing that the hard part's over, and we can just have this do the easy part, which is going to cost us way less. So I'll just say, make a research paper outline based on this. Again, my typing is terrible today.
- 24:58
But here we go. GPT OSS going to write the whole outline for us, and it's going to do a great job. Again, even the dumb models today are fairly good- Yeah ... especially at simple, straightforward tasks like this. And there we go. And again, Magai making it easy for you to switch between these models is one of our founding ideas, is give people the options.
- 25:27
But I also know that sometimes decision fatigue can be taxing. How I've got to look through the list and know which ones to pick. That's why we created the auto model. So, if you are the type of person that's like, "I don't have the time or energy or mental capacity to try and figure out which model I should use when," just pick auto. Auto is there so that you don't have to do the brain work. Based on your prompt, it's going to pick the best model for the job. And so if I were to say, okay,
- 26:01
turn this into a full finished article to publish on my blog. Auto is going to take that, interpret the prompt, and it's going to route it to the model that it thinks is best suited for handling this task. And many times, it'll pick a cheap and efficient model.
- 26:33
Sometimes it'll pick the more complex, expensive ones that you never really know until it's finished. But see here, it actually chose GPT 5.4 Nano, which is one of the cheaper models. And it looks like it did a fairly exhaustive job here. Pretty long blog post article. And all of that to say that I think as we hear more and more about these new models and new capabilities and new features, and again, with all the pending sort of cost rising,
- 27:06
I think you, as the user, are so much better equipped than everyone else when you understand how these costs work. And when you understand the underlying nuances, the underlying complexity, and more than anything else, you know that you don't have to just use the smartest, biggest one. You can leverage them against each other. Utilize the complexity and intelligence of the highest model, and then immediately switch to a cheaper model to finish the job out. Or vice versa.
- 27:41
Do all the dirty work, the initial work with the cheap models, and then only bring in a smarter model when absolutely needed. When that nuance that you're looking for or that level of sophistication you're looking for isn't quite there, just flip it over to a smarter model, see how that model handles it, and then you carry on from there. And that's how we become not just AI consumers, we become AI masters.
- 28:12
Marlene has a question, and it's a phobia, I think. I always feel I have to rush that AI is going to leave me or get tired of waiting for me to get back to what we're doing. Interesting. Isn't that funny? Yeah. Yeah, I'd love to know some examples of that.
- 28:37
Well, no, I think what she's saying is that when AI responds, she tries to respond really quickly because she thinks, "Oh, I've got AI's attention now." Oh. I got you. You can come back three days later, and it will be in the exact same place. Yeah. So this is an anthropomorph- Anthropomorphizing ... ization. Yeah.
- 28:58
That's a word of AI that I think a lot of people go through, too. But literally, the AI has no concept. It's not sitting there waiting for you. Yeah. You send it a message, and it replies. The AI is not conscious. It is literally just reacting- Doesn't have- ... to anything you do to it. So- Yeah. It's got infinite patience. You could ask it the same question in multiple different ways if you don't get the answer you want. It doesn't get tired of answering the question.
- 29:26
And you don't have to worry about hurting its feelings. If it's giving you results that are not anything near what you're expecting, you can say, "No, this is not what I'm looking for." And you can say it as rudely as that. Yeah. No, this is not what I'm looking for. This is what I'm looking for. And it doesn't bother it. But Marlene, I think you're not alone. I think that the AI that we- Yeah ... I always say please and thank you.
- 29:49
I've just- Yep. So- Now, one thing I wanted to note before I forget, I almost forgot this, is in Magai, we've also added a new tool. If you click on the information panel here over on the right-hand side, you scroll down, and you'll see there's a conversation size section that shows you exactly how large your conversation is. Right now, looks like we're around 5,003 tokens for this conversation.
- 30:15
And it'll actually show you based on the AI model that you currently have selected or the one that was previous last used. So again, we were over here. We used GPT 5.4 Nano. It's showing that you have used about 4% of the context of that model, the last used model. And we have built in a feature that allows you to compress these conversations. So like you were saying before, Steve, you might have it summarize the conversation so you can go start a new one. You don't have to do that anymore.
- 30:46
Inside Magai, we have a compression feature. Now, the caveat here is you have to have at least 11 messages in the conversation, because what it's going to do is going to take all but the last 10 messages, and it's going to compress all of that other stuff before those last 10 messages into an easily digestible format that's going to be a lot more concise, doing its best to maintain all the important bits.
- 31:13
And then you're going to minimize that conversation going forward. So it's almost like it cuts off the previous messages and summarizes them for you, and then you start with that summary and those last 10 messages to continue that conversation. So that's all built right into Magai. It's on by default. You can turn it off, or you can even lower the threshold or raise the threshold for how many tokens you want it to compress at. But that's just one way that we're trying to help users with that exact problem is, maybe I do need to have this conversation be really, really long, but I want to find a way
- 31:49
to maybe cut the cost or reduce the waste. This is one way you can do that. Okay. We have a couple of questions. I think Alex is mistaking the concept of how you apply multipliers. Do we get a monthly allotment of usage multipliers? Do they carry over to the next month? Can we pay for more as needed? The multipliers are just a reference.
- 32:13
It's a reference to understand how expensive one is compared to the baseline. So we've got the baseline, which is one, and then everything else is just being compared to that baseline in terms of how much it's going to cost. When you pay your Magai subscription, let's say you pay $20 a month, you get $20 a month in usage. That's it.
- 32:36
It doesn't matter which model you use, each model is just going to consume more of those $20 in credits than others, more or less than others. And so it's not the multipliers that are really doing anything, it's the actual cost that we're calculating. The multipliers are just there for you to be able to visually reference, is one more expensive than the other? Is one more cost-effective than the other? That's what the multipliers are there for.
- 33:04
Avi's asking, "Is there a way for the end user to see which model Magai is using when we choose auto?" Yeah. As I demonstrated here, once it's finished with its response, it then relabels that message with whatever model actually handled the job. And so- There's no way for you to tell before. Because it has to process it, that's why. Yeah. It has to process it first, and it's just an automatic process.
- 33:29
It takes your prompt, categorizes, decides routes, and it's all part of the same sequence of events. I think this next thing is going to become bigger and bigger as we go, I think. So Mike is asking, "What do you think about the local AI solutions coming with the M4 and M5 mini Macs and running local LLM machines rather than in the cloud?
- 33:53
This plus RAG should help in cutting cloud token costs." And- Yeah. It'll definitely help in terms of- "... how will Magai work?" It'll definitely help in terms of costs. The problem, of course, is that most of the tooling, which are the interfaces built around these models, are most likely not going to be able to connect to your local models because there's so many variables.
- 34:20
You'll have to essentially rely on whatever the local UI wrapped around that model is. And so, something like Magai, it's a web-based app. And so there's no way for a web-based app to connect to the core files and systems and apps on your desktop. You would need a custom desktop app to do that, and we're in the process of building that. But it's a very difficult- So where is this technique going to be valuable?
- 34:53
Well, the other thing too is usually local models are going to be less powerful. They're always going to be less powerful than the newest- But more specific ... models. Right? Less powerful, but more specific. So as a- Could be ... for example- It could be ... will companies build a knowledge base over time and keep that locally? A legal office keeping, maybe there'll be a RAG of legal documents that, kind of a library of documents that they can load onto their systems. Local storage is very
- 35:26
inexpensive now, so they can store massive amounts of data. Mm-hmm. So, I realize that I'm just kind of throwing this out to you without having prep, but there have to be some use cases that we're going to see that this concept- Oh, yeah ... is going to be really relevant for. A lot of companies who are overly protective of their maybe proprietary data, and overly is the wrong word.
- 35:51
They are very strict on their- Mm-hmm ... protection of proprietary data, or just internal systems and so forth, that they don't want any risk of getting out into the internet, then yeah. Totally great use case for them to make sure that everything stays local in their network. But, for the average person, I tend to think of the average person, not the corporate entity.
- 36:16
Mm-hmm. Enterprises and corporations are already doing this. This is not new. They already- Yeah ... have these systems in place, but for- It's just we're starting to see it offered on individual computers, like the Apple. It's really one of Apple's initiatives, and I imagine it's based on privacy, really. Yeah. But I think what we'll find is very much that what people want is they want it connected to all of their things, right?
- 36:43
And that's the demand that we're seeing at Magai, is people want the AI connected to their Gmail and connected to their- ... their ClickUp and their Basecamp or their- So- ... all their different tools. And a local AI is going to struggle with that because these local AIs are going to be less powerful, unless you can afford the super $5,000, $8,000 rig on your machine, which the average person can't.
- 37:14
They just don't do that. So- I don't see a whole lot of practical use in the local LLMs Got it. Okay. So Dustin, I know you're probably thinking about this and without giving away too much internal research, what do you expect a year from now the average account is going to look like? For example, when we started, the very first days, everybody was free.
- 37:43
We all had free ChatGPT and the standard account that we all had was free. About four months later, those of us that were using AI said we got to pay, and $20 was the ticket. That's what it cost to enter- Mm-hmm ... the game. Your pricing was based on that model, ChatGPT, Perplexity, they were all based 20 bucks a month.
- 38:07
That was the going rate. And nobody was more and nobody was less. It was 20 bucks a month. Yep. Today, 20 bucks a month, I think is still the standard account that most of us have, but it is rapidly starting to show cracks. What do you expect the standard ticket of the future, six months, eight months from now, is going to be?
- 38:32
Well, six to eight months is not a lot of time for human behavior to change. But I can see- In AI world, it's like seven revs. Yeah. Even then, humans are hard. And I still think most people, they don't really grasp the value that AI gives them back. And so because we've set this sort of price precedent of $20 a month, I think a lot of people are going to stay there.
- 38:59
And they're going to run out of usage and then they're going to go, "Well, I can't pay anymore because $20 is sort of the ceiling that I put on myself." And they haven't really evaluated what that $20 is actually buying them back in terms of time or energy. And they haven't found a tangible ROI from it. So I think a lot of people are going to stay there at 20. Mm-hmm. But what I'm already seeing in Magai is the people who are serious, the people who actually get it, the people who are actually using AI for real work scenarios, for real return on investment,
- 39:32
building landing pages and all of a sudden the conversion rate is twice as high, right? That example that I used in the past. The guy who's using it to build his slide decks for his keynote speeches, right? Getting paid $1,000, $2,000, $10,000 for a keynote speech. Having the AI do that for him is a critical ROI on his time and energy. The people who value their time and energy and are doing real work with the AI, not just fiddling and testing and shortcutting things that don't really matter, they've already upgraded to the $40 plans or the $200
- 40:06
plans. The people who are serious understand the value, and they are already upgrading in droves. Let's- While the average person is just, $20 seems to be their ceiling at the moment. Just those of you that are online right now with us, if you have a threshold that you're willing to go to, you're comfortable. If it's zero, put in zero. But tell us in chat what your kind of the threshold is that you are comfortable going to.
- 40:34
Personally, I think $100 right now is kind of where I hope that I can get away with. Yeah. Because I do see the value. I build web page- Yeah ... I just built a massive landing page, web page, that took an entire weekend of AI. I used up my entire account of Magai pretty quickly with it. Yeah. But I'm thrilled with the outcome. It was- And you can easily- And if I'd hired a web designer to do it- Right ... it would've been very expensive.
- 40:59
See, you can easily make that connection to, well, the only other option I had was to hire a web designer who would've charged me- Yeah ... at least, what? What's the going rate for a single web page now? Like $600, $800 maybe? Oh, I have no idea. Yeah. And the amount of patience that I require to explain to them what I want back- Right ... and I have to do it politely with a web designer.
- 41:24
Yeah. And I have to be cautious of their feelings. Right. Actually, I will tell you, I used a combination of Magai for the copy, and then I used the new AI features that are built into Leadpages for the layout. Mm-hmm. And it ended up being great. A lot of AI usage, but it ended up just terrific. So looking here. All right.
- 41:49
Yeah. So we have a lot of our- So I'm seeing a lot of $20 and a few- Yeah. Few hundreds ... hundred dollars and here's the other thing that keeps coming up in my mind as I'm out doing my daily business. I think the mindset change has to be somewhere along these lines. When was the last time you said, "I've already spent $50 in gas this month.
- 42:14
I'm not going to spend another 50." I don't think a single person in this chat watching this has ever said to themselves, "I've already spent so much in gas, I'm not going to go anywhere else this month." Yeah. We can't go anywhere. We're not going to drive the car anymore. Yeah. Because gas prices- But we don't quantify, right? We don't quantify what that gas mileage we got out- Ceiling budget ... of it returned to us, right?
- 42:36
Yeah. We just know we have to go places. We have to go do things, and the cost is putting gas in the car. And I think that's the shift mentally we need to make. If the AI's producing results, if it's getting you to destinations, the cost doesn't matter. Of course, we shouldn't be reckless. Shouldn't be speed racing from here to there.
- 42:58
Again, not using the Ford F-150 to go do things. But when did we ever put a cap on our... Okay, so there was a time in my life where I was seriously broke. And I struggled financially, and I did have to budget my gas, right? Yeah. And so, maybe that's where a lot of people are. Sue just said, "I have half-filled my car, though." So- Yeah ... but you still put the gas in, and you just- You still put the gas in. You said- You're robbing from Peter to pay Paul ... you did manage to get to the car.
- 43:27
Yeah. You budgeted that, right? Yeah. And so to me, that's what AI is. AI is just a vehicle. It's a vehicle- Yeah ... to get me from point A to point B. And I think the rest of culture is on that path. I think it might be two to three years, maybe five years, before people get to that point where they realize that.
- 43:50
But right now it's just ChatGPT and all the others have put this sort of $20 idea in our heads, and it's like a ceiling to us. Can't get anything more done after I've hit that $20 ceiling, and I think we need to get out of that mentality. Think of it as a vehicle. Well, and that's when the penny drops and you recognize the value, and you recognize the return that it's giving you.
- 44:12
So your business has to be in a place where the results are... And- Yeah ... in many cases, it's also building that level of trust to change an existing process. If you've hired somebody in the past to do your design work, and now you're going to take it on yourself with Canva, ChatGPT, and a few other tools, and you're going to take on the responsibility yourself- Mm-hmm ... you have to see the success. You have to have a couple of wins under your belt before you're comfortable, and then you realize.
- 44:45
Because in the scheme of things, it's really, really inexpensive help. It really is. Yeah. It's so cheap. Ridiculously inexpensive. But it's also done in a vacuum. You do it yourself, you sit there in the screen, you look at it, and you go, "That's what I just paid for. Is it good enough?" And you are biased one way or the other because it's you. And there's also, let's be honest, especially if you're doing, say, graphics, there's the AI bias.
- 45:14
You look at the picture and go, "That picture's exactly what I want, but it looks like it's AI." Mm-hmm. So is it going to do the job for me? We don't know whether or not it's going to work yet because we've got this one side of us saying, "Oh, I like that. That's me looking." Yeah. I did an image of myself as a doctor. Looked great. Uh-huh.
- 45:36
Did exactly what I wanted for the webinar. But at the same time, I'm going, "But it's AI. Is there real value?" But I had to- Right. Well, just along that same line, Steve, I probably said this in one of our image-based academies, but as a designer and as a blogger, I would spend 20, 30, sometimes 40 minutes looking for the right images on all the stock image sites.
- 46:00
20 to 40 minutes of my time, and I've always valued my time. To me, I always had in my head, "I'm worth this much an hour. Whether or not people pay it, this is how much I think my time is worth." And that 20 to 40 minutes was costly. Now I can just type it out and- Yeah ... it costs me $0.06, $0.12 per image on average.
- 46:25
That's right. And if it's 10% less effective, is that worth it? Is that trade-off? 100%. Here's something that shook me and I haven't figured it out yet. I have a clip art or a clip media subscription. I've had it for many years. It's called Envato. It's a great service. Mm-hmm. Costs me a couple of $300 a year and I've used it extensively. Stock photography, stock video. I use it in all of my kind of sales type webinars. If there's a little bit of B-roll video, it's coming from Envato. For this
- 47:00
last landing page, I went to Envato and I said, "Okay, I need a picture of a middle-aged person at work being marginalized by ageism." Boom. I put it in and it comes up with this typical series. I go to download it, it asks me what format do I want it, do I want to extend it, because it's going to AI it for me before it uploads it to me.
- 47:24
Hmm. That's interesting. So I couldn't get the photo. Hmm. I got the photo base with the AI treatment with some images. Isn't that interesting? That is interesting. Yeah. As I said, I don't know what this means yet. I haven't figured out what it means.
- 47:48
Yeah. And as far as my subscription to Envato, is it now something I go, "Well, I only want it for real photos. I want it to look real." Yeah. So if I'm going to have an AI photo, I've got lots of pictures of my face. I can make myself look like everything from the captain of a steamship in the 1800s to a spaceman. Right. I can do that in there. So if it's going to do that, I'll just use me. Right.
- 48:11
But, yeah. I want to point out real quick what Sue just said because I think it's so true. She says- Yeah ... "I think people have to get out of the AI is just a Google search with more stuff, so it should be free." That's the mentality a lot of people have. They just see it as a Google search that does more stuff. And you're absolutely right, Sue. People need to get out of that. They've gotten used to Google searches being free, and so they expect their AI search to be free and that's just- And now when you do a Google search, the first thing you see is an AI response Yep. That's true.
- 48:46
Now the other thing I wanted to say real quickly too is I'm not an advocate for just spending lots of money on AI. In fact, I'm pretty much the opposite, and most of this lesson has been geared around to how to teach you not to overspend on AI, right? Yeah, I think people should be okay with spending $100, $200 a month on AI, but not recklessly. So I have a lot of developer friends, highly technical friends, who when all this AI agent stuff came out, the OpenClaw and the Hermes agents and all that stuff, they went bonkers on it. And they went from paying $20 to
- 49:22
$40 a month for their AI subscriptions, but now they have these AI agents that can work around the clock for them, and it's going to unlock all these things for their business. And almost overnight, they went to spending $200 to $400 a month on these AI agent tools that work nonstop for them. To what end?
- 49:44
Zero extra dollars in revenue. And so I think the big thing to be afraid of is being so enamored with what is essentially AI busy work that is not driving value. And I wrote an entire blog post and ranted about this. I'll put it in the chat there.
- 50:07
How AI can oftentimes convince us that it's giving us value when it's really just a bunch of nonsense work that is wastefully consuming our hard-earned dollars. And I'm a very big fan of tying value to the AI work that you're doing. And so, for anyone who is conscious of what they're spending, good, stay conscious.
- 50:33
But always try to understand what value the AI is bringing you, and if it is bringing you value, you should be okay with putting more gas in the tank. Okay. With that, good conversation today. Educational. We all have had our eyes opened a little bit and learned something. All except Dustin. Dustin didn't learn anything, but the rest of us learned something today. So thank you so much, Dustin.
- 50:57
To each and every one of you, thank you for the time. Thanks for the input. As usual, our community is just awesome. I want to wish all of our southern neighbors a happy 4th of July next week, and to my fellow Canadians, happy Canada Day. Coming up next week, we will see you all back here for AI Academy in a couple of weeks. Check your email, check your messages. We will see you then. Until then, have fun storming the castle.
