AI Agents vs Automations: How to Choose

Today, we're going to be going over automations and AI agents. If you don't really know what these things are, but are interested in learning about them and using them, this will be a good video to start. If you are a current user of automation or AI agent products, but you find that they're unreliable, you can't get them to work, or that they're just costing you a ton of money and you're not seeing the results you want, this will also be a good video where we're going to explicate the most likely reason that is. The most likely comes down to not understanding the difference between these two things or using software that obfuscates the difference. By the end of this video, you will know enough to go forth and master and use these tools in a way that's cheap, reliable and gets you the results you want. Let's start with what an automation is and specifically a business process automation. It's some sequence of events that runs without human intervention. One way to think about it is as a bunch of if then statements. So if the email is from Joe, then label the email as finance. If I finished the call, then append the notes in Salesforce. And here's one you've probably used before. If I receive an email from now until next week, then send out the out of office reply. These are some pretty simple examples. But the basic idea is you have a set of decision rules. This would be one of them that get followed every single time a certain event happens. The event could be a set schedule. So every five minutes, something could be triggered. The decision tree might also have branches. So if the email is from Joe, it's labeled as finance. But if it's from Sally, it's labeled as catering. In many products, including Carly, these automations are called workflows. One property of a workflow is that it is typically deterministic. We'll get to cases where that's not true later. If something's deterministic, it means that if you run the same thing twice, there is only one possible result. If it's from Joe, then it gets the label finance. If I run it again, nothing changes. It gets the label finance. This means that as long as the decision rules you write are good, you will be able to guarantee an outcome. Workflows are also incredibly cheap to run. This is because they don't involve LLMs, large language models like ChatGPT or Claude. The main driver of costs when we're talking about automations and AI agents is LLM costs, large language model costs. If you've ever used a product like ChatGPT or Claude, this is a large language model. And the reason they're so expensive is because they produce tokens and the model companies charge for those. But with deterministic workflows, there are no large language models, no AI. This means that the cost is basically free for the user to run. If you're a company that hosts workflows, you're of course paying for server costs for your engineering team. But this is exactly what allows certain workflow companies, Carly AI included, to allow users who don't pay a penny to run their own workflows. The biggest con is the setup time. To have a really useful workflow, there are typically a number of steps and a lot of them you won't even think of. Let's go back to our out of office example. Let's say that on September 16th, Joe sends me an email, it says hello, well, what happens if it's within this timeframe, which it is, my system will respond, Sarah is out of office. Now the next day, what if Joe replies hello again? Now if this is the only part of the workflow, then that means that Joe will get another email back saying Sarah is out of office. Now this might not seem like a big deal, but consider the following. When it comes to email, there are ways of spoofing emails, meaning Joe can write me an email that looks like it's coming from the president of the United States. So what if Joe takes a random email, maybe it's that of the president of the US and writes an email to me with some scammy thing in it. Now if our entire workflow is just this if then, that means that the president of the United States is going to receive my reply, Sarah is out of office. But even worse, it might actually quote in the reply to the email this language. And now all of a sudden Joe can send me a thousand emails from different accounts that Joe is pretending to be. And suddenly my email is telling a thousand different people that they've won a free cruise and I'm unwittingly involved in this scam. What this is meant to show is that our simple workflow here is really incomplete and there's always more than meets the eye when it comes to workflows. So in this case, the way to prevent something like this from happening is number one, you can have a rule that it has to be, Joe has to be a member of your contacts to reply. You're not going to get an out of office reply. This means that if Joe tries to spoof the president of the US with that email and all the unwitting strangers that Joe is spoofing would not get replies assuming they're not in my contacts. The second way to get around this is you can have rate limiting. So maybe after a hundred different out of office emails, something must be wrong at that point and the system will just stop sending them. Finally, instead of saying Sarah is out of office and then including this reply text down here, we might just have the email not have the reply quoting. So in the case that Joe writes you've won a free cruise and it gets sent to a bunch of people, at least it's not going to include this line. It would just say Sarah is out of office. Now this is a bit of a silly example, but for each one of these, there are a bunch of conditions that people don't normally think about and can end up biting them. I'm also going to include in this some technical know how it's not strictly required to set up workflows, but having some experience setting up workflows in the past and knowing how to think like an engineer can help you anticipate some of these things that might bite you when you start building out the workflows. So now let's move on to AI agents. There's a lot we can say here, but the simplest way to get your head around AI agents is it's a set of prompts, which are instructions and then tools, which are things the agent can do. So here's an example of an agent prompt. You're a sales agent, you work for this company and then you would include the company name and probably a bunch of information on it. Your goal is to qualify the lead and send a Stripe link when they ask to get started. This is actually pretty close to my introduction agent. Now what tools would an agent like this need? Oh, we need Stripe. Stripe is a payment processor because it would need to be able to generate a link that would allow a person to check out. Here I'm actually only considering the case where there needs to be some dynamic input. So maybe the agent is going to negotiate with the prospect. They'll arrive at some number, some discount. And then the reason it needs this tool is the agent needs to go to Stripe and say, can you create a checkout link where the price is $50 a month? If there's just one set price and a set number of seats that someone is buying, you wouldn't actually need a Stripe tool. You could just put the Stripe URL right here because it's the same for everyone. And then the agent will just give that when the person is ready to check out. You might also want to connect this to your CRM. So what if the person that is emailing the agent already has an account and has already paid? You don't want the agent to resend the payment link and have them double pay. We also need an email tool because the agent is emailing. It could also be texting the prospect to communicate these things. We might also have a calendar tool. If we want the agent to book meetings with certain prospects, it can check people's calendars and then offer times and create events. If you want to learn more about agents and the use cases, by the way, if you go to our website, we have a bunch of resources, including templates, prompt library and a bunch of use cases so you could get started and also expand your mind to the types of things that are out there. And same for workflows. As we were talking about before, there are a bunch of prebuilt workflows here so you can just get a sense of the types of things people build. Let's talk about the pros of using AI agents. They're very quick to set up. There are a bunch of different ways to set up agents. The easiest way using Carly AI is either going to the email agents tab and pressing create new specialized agent, or you can just copy the Carly email or whatever you renamed Carly to be. And Carly is an AI agent that's already built out and you can customize by adding your own integrations, but here you don't need to configure anything. So it can just be as simple as emailing this email address. You can also email Carly and say, can you make me an agent named this? And these are the tools and the abilities it should have. I will link in the description a video I've done on this showing you how to make an agent very quickly if you want more detail. But this is all to say. It's very quick to set up. There might be no setup at all. You might just be emailing Carly, for example. Next pro is what I'm calling adaptable. I'll explain why the quotes. Let me give you an example of what I mean by this. So imagine instead of setting up this workflow, if the email is from Joe, label it as finance. I said to Carly, my AI agent, every hour, I want you to look at my emails and if it's from Joe, then label it as finance. Also, if there are other emails that are not from Joe, but they look like they're related to finance, go ahead and add that label to it. So here what we did is instead of being forced to define all the conditions that would lead to a certain action, like who was the email from, we can just say, hey, use your judgment. And then the agent, when it wakes up to run in an hour, will go and read all the emails and think, is this finance or is it not? Now, the reason I have quotes in here is that anything an AI agent could do really can be done with a workflow if you add some LLM step to it, which we will get back to. If you don't know what that means, do not worry. But it would just take a trillion years because there are so many different cases that a human could not possibly write out every single one. If you were able to predict the exact copy of an email, meaning if you knew, maybe it's not Joe, but you could say all the different possible words in their combinations that could come into your inbox such that they should be categorized as finance. That would be awesome. But that is probably an infinite list and it would be a waste of your time to try to write all of it down. It would be much easier to say to Carly, hey, if it looks like finance, label it finance. Now the cons are, this is very expensive to run using AI agents, which are running these AI tasks where there's prompts, there's tools, there's tokens being used. It is just like using ChatGPT. It costs a lot of money. It could also be really unreliable. AI is not deterministic, unlike in the deterministic workflows case where we just had if then and if you put the same email in twice, you get the same result. This is totally deterministic. There's no probabilistic outcome where we don't know due to randomness whether or not it will be labeled as finance. We know we set it up. It will be labeled as finance with LLMs. That's AI. On the other hand, it is totally probabilistic. So imagine we run the agent once and the first time it labels five emails as being related to finance. There are two receipts. There's a cash flow statement and two CFO emails. Imagine we ran the same agent again. In this case, it only labeled three of them as being related to finance. The cash flow one, just one of the receipts and one of the CFO emails. This is a very common occurrence because it is probabilistic what the AI, the LLM decides to do. In this case, it thought receipt one was related to finance and here it didn't. Now this could be pretty disastrous in the case where it's not just a matter of labeling. When you say to the agent, you should forward any finance related emails to my CFO and as part of the tool is it has your access to your email. Now what if the LLM accidentally classifies a really embarrassing email as being a finance email? Now suddenly the side effects are pretty severe, you're going to have the really embarrassing email sent to your CFO and this is because you are out of control. There is some nuance here that I am skipping over and if you watch one of my email agent videos, we go into it. The short version is that when you create one of these email agents, there are ways of restricting what it can do. You can block it from responding to people that the owner did not approve of. You can also pick which specific tools the agent has access to. But of course, the more of this type of setup you do, the less true it is that it is quick to set up. So frankly, most people are not very thoughtful when they start building AI agents and it does get out of control and then they need to spend all the time reining it back. Next, we're going to move on to how you can have an AI component in your workflows. But I first want to make a quick point about terminology here. We've been talking about AI agents. There's something we have on Carly AI called tasks. What a task is, is not a deterministic workflow. It's another agent that wakes up at a specific time. It has specific instructions and tools. And then using those instructions, aka prompts and tools, it will run, it will do its thing generating tokens, making tool calls, and then it will die. But when you hear tasks and when you're using tasks, you should think this is really an AI agent, but it's a special type of AI agent that has woken up at a certain time in the future. I didn't trigger it via emailing it or texting it. I instead, at some prior point, talked to my agent said, Hey, in a month from now, can you look through my emails and find this thing and send it to me? That created a task and then when the task occurs, the agent will wake up again. Now it has a prompt that says go look at Sarah's emails and find this one thing and then send it to her. And then when it's done, it will die. Now back to deterministic workflows, aka traditional automations. Carly workflows, that tab, there are a bunch of different workflows. A lot of them are labeled no AI. This means they are the traditional workflows. They're deterministic. These are just the simple if then statements. But some of them do not have the no AI thing. What this means is there is an extra step in the workflow where there is an LLM called. Let me show you what this looks like. So I've just opened one up. It's called triage and file every email. What this does is you select whatever emails you want it to look at and then you send it to an LLM and you give it a bunch of instructions. You say, this is my name. This is kind of what matters. So here are the possible labels you can use and I can just copy all the names of the labels that I have. You're telling the AI, I don't invent any labels, just use the list I have. And for each email, tell me where you think we should file it. And then you say, by the way, here's the email. And what the AI should give back is the destination, which is the name of the folder or label. And then you can also have a reason. And we also have a line in there saying, if you're not sure, say unsure. So keep it in my main inbox, I don't want you to file it anywhere. And then I'll take a look myself with this most recent email. The destination is unsure and we can actually see the reason. So if this kind of speaks to us and we say, oh, actually we want the bar association emails to be considered, I don't know what my labels are, but promotional, you can then go back to the instructions and say, if it's about any association, it should be in the promos. Now after the classification happens, then there's an if branch and that should sound familiar because it's if then. And what this is saying is if the AI found a destination that was not unsure, that was, I don't know, promotional email label, then it will file the email into that folder or label. If it's anything else than that, it does nothing. It ends. It doesn't get filed. And then here, step five is the file email step. Now do not worry if none of that made any sense to you. You will never have to go into this much detail or ever look at this page ever. This can all be configured by talking to Carly. But the point here is that not all your workflows need to be deterministic. You can have a data classification step that says, is this a promotional email? What is this about? What folder should it go into? And now you could do a lot more. Let's go back to the Joe example. So if it's, if it's from Joe's email, then say he's the CFO, it's going to be labeled finance. That's great. Now we could do the following. If it's not from Joe, we're not just going to leave it now. Now we can actually add this AI step, send it to an LLM, send that email body to an LLM and classify, is this email about finance? Then the AI will say yes or no. If it is a yes, label it as finance. If not, leave it as it is. Now you might be asking, is this going to make this way more expensive to run? Because thought the whole thing was it was cheap because there are no LLMs and now you're telling me you can actually add AI to workflows. And the answer is it's still pretty cheap because if you notice this step is really small, there's a prompt and it's pretty short. It's just looking at one email and whatever instructions you had above. There's also no tools that saves a lot of tokens. The only thing that the AI is returning is in this case, we're saying give us the folder destination label destination. Tell us why you don't even really need the reason one. It's just to help with auditability. This is a super small task. It can use a super cheap model. And the reason it's so much cheaper on a less technical level is it's doing way less than your agent was. The agent has a bunch of tools. It has a bunch of instructions because it needs to deal with anything you might ask it for. It could be scheduling. It could be web research. It could be adjusting your CRM. This little task is just classify this. It's very little work. It's not really doing anything. It's just outputting the destination and a reason and that's it. The best part is it's also still very reliable. Why? Because you're still setting up the conditions exactly as you want them to happen. First of all, in the case where you want Joe to be labeled, he's labeled. It's better than with an AI agent, an AI task setup. You're not sure if the AI will remember that Joe is your CFO and it should be labeled finance. But here we do have the deterministic step to make sure at least that's good. Now for this step, it's true that if you run it the same thing twice because this is using an LLM, it might in one case say this random email is finance, but another time it's not. So that is still a risk. But at least you control the prompt that is used to decide. So if you see it keeps messing up a certain type of email or it's inconsistent, you can go and tell Carly to fix the workflow or go into here yourself and say, hey, if it's about this thing, it should always be classified as finance. You control the flow so much more, even though there's a level of determinism because it's such a small job, you can get very fine-grained detailed and make it as reliable as you can. And now remember the case we were talking about where your CFO gets the really embarrassing email. Well, what we can do here is we could still have it be sent to the CFO, but we can add an approval step saying before you send anything, you have to email me and get my approval. So this way you can review every single thing. Let's review quickly what we've gone over. We've talked about automations and AI agents. Automations are workflows. They're mostly deterministic. In some cases you might have a little AI classification task, but it's mostly if then. They're really cheap. They're very reliable because you know exactly what will happen, but they are set up heavy. AI agents are the opposite in many ways. They comprise of agents and tasks that your agents set up that wake up a new agent. They're super expensive to run, but they're very adaptable. They can do anything and they are insanely easy to set up. Let's talk about some common problems. The first and by far most common is you have an agent or a task, like a recurring task, but it should really be a workflow. Let me give you an example. Maybe I say to my agent, hey, every hour go through my emails and label the emails accordingly. Then the agent says, okay, I'll do that. It sets up a task. Remember a task looks like this. Some time passes an hour passes. Now another agent wakes up and then it runs with all the tools and the prompts that we gave it and then it dies. Now that might work great. The agent might get it. It reads all your contacts and understands where things go. This will cost you a fortune because when you use an agent, you're using a ton of tokens because it's holding that long system prompt with all the instructions and a bunch of tools that it doesn't need actually to label emails. What you should do is say to your AI agent, hey, make me a workflow and the agent will be smart enough to know it needs an LLM step, the step that will classify the messages. Every time I receive an email, it should be filed accordingly. By the way, we have a bunch of these workflows that are pre-built so you don't have to worry about remembering any of this. You can go and copy the prompt, send it to your agent and the agent will set up a workflow for you. Then this workflow will be so much cheaper because there is an LLM involved, but it's a very small data classification task. If you feel like you are burning money on tokens, the first place I would look is seeing if you can turn your tasks into workflows. Another problem is not spending time tweaking your workflows. Workflows are deterministic rules mostly with a little LLM, but as you saw, even with the LLM classification, that instructions thing will need tweaking because it might not understand bar association events mean it's promotional. There's a principle in computer science, garbage in, garbage out. Your outputs are only as good as your input. If you look at a workflow and say, why is it labeling all my emails as finance, for example, it's because your inputs suck. It means probably your instructions are wrong or your deterministic rules are wrong. Maybe it's saying label all emails finance. Maybe the instructions are saying, if you're not sure about what an email is, you should label it finance. That's garbage in. Then of course, all the labels will be bad. The last one is a catch all and cop out, but I see people get tripped up when they're not willing to just slow down and take a second and think about what needs to be true for this to be successful. For example, maybe I set up a workflow that will text me every time I get an urgent request. Are you thinking about the types of emails you're getting because you might be getting things that look urgent, but are actually food delivery notifications. Try to use your brain a little bit. That's always good advice and get excited because now anyone can really build one of these workflows. It used to be super annoying to do. You used to have to use things like Zapier and drag and drop and you'd have to do all this logic and it just sucked. Now with AI, you could just email Carly and say, "Can you set up this workflow for me?" Take a look at some of the resources we have down below. If there are things that we didn't cover that you want me to cover, you can go and reach out to my team. You could also book a call with us if you want some custom implementation and things done for you. We could see if we can help. Thanks for watching and I can't wait to see what you build.