As we go along, more and more features keep getting added to these tools, but their full potential is still unknown, including the AI ones. I think people are starting to ignore these assistants, since early uses have shown they’re very rudimentary and not extremely useful.

So as I’ve gone along in this journey of figuring out what tools work and what tools don’t, I have made some observations, and I’m going to give you the rundown: what things actually matter, where it falls short, and what’s going to cost you long term.

What actually matters

My definition of useful here is that anything that helps you save time or money in your business. That’s the biggest indicator of whether a tool should be on your radar or not. That looks different for everyone, but there are a couple of ways we use it today that can help us.

The first one is drafting things. Proposals, emails, job posts, policies we have to write, graphic design content, and even figuring out how to word a specific message for a specific audience. It takes you a good portion of the way, but it doesn’t get you to the finish line unless you want to accept whatever AI delivers to you. This isn’t a good idea. A lot of people do this, and because of that, the digital world is filled with what’s called AI slop.

Another helpful thing is pulling answers from your own data, if you can feed it in. For instance, we pulled financial data because we could get our AI models to connect to our financial systems. We asked very simple questions, like which companies are costing us more money based on the hours we put in. We asked which vendors have been slowly increasing costs on us without us realizing it. What’s our average cost of goods percentage year over year? Things like this help us figure out where we need to make adjustments, without spending a day on spreadsheets or making data-entry mistakes.

If you want to try this yourself, you can type things like:

“I run a [type of business]. Ask me ten questions about how my money is spent in my business, and tell me where I’m losing money, year over year or even month over month.”

“Here’s a vendor quote. Give me in plain English what this quote is, flag anything that doesn’t look right, or terms that don’t stand well with my organization.”

For instance, if the vendor is locking you into a five-year auto-renewing term, that’s something you’d want it to flag.

But just as a note. Before you put any data into an AI model, make sure that it’s private, and that they don’t train it on what you give it. Usually that means paying for AI. Paying for an account with one of the big providers and reviewing the contract terms helps ensure your data isn’t being sold or used for training.

Where it falls short

The biggest thing I’ve found is that it does a couple of things really poorly.

First, it’s super confident in its answers, which can give you a false sense of truth. Whenever it gives you answers based on data, I’ve caught it making some of the data up. I have gotten flat-out fake data when, in reality, that data didn’t even exist. Consider challenging the model if something feels off, and give it data to prove you know what you’re talking about. Otherwise, it’ll take advantage of your blanket trust of the model.

The other thing I’ve found is that AI can actually make changes if it doesn’t understand what you want it to do. I’ll give you an example.

We had AI make some changes to one of our systems, but it didn’t fully understand the format it needed to push. So instead of updating a couple of line items, it wiped a big chunk of data for a sales order process. It didn’t give us any confirmation. Should I do this? No warning. It was very confident it would make the appropriate push to the system. When we found out it deleted everything, it made us realize how cautious we have to be about letting AI make changes.

Now, a little side note. We had everything backed up, and we always have a back-out plan before we let it touch anything. Also, the data it touched was internal data that didn’t really affect the business, and we instantly rolled it back, which was part of the plan anyway.

The ultimate problem is that the AI didn’t know what it didn’t know, and it didn’t stop and ask, Hey, are you sure? It had the data, it had the parameters, and it thought it had what it needed, but it never tried to dig deeper or ask us to do a little more work to prove it was good. Keep in mind, some of that was also because we didn’t prompt it properly. We could have said: “Before you do anything, make sure you’re certain about the change you are going to make, and verify it with known documentation; otherwise stop and present your facts and allow me to make a final decision before pushing a change.” Ultimately, if we’re going to trust AI as a business owner, not as a technical resource, we want it to do the task when we ask it to. We don’t want to quadruple-verify that it knows what it’s doing; otherwise, we could do the task ourselves.

The hidden costs

Like anything, nothing is free, and using these built-in AI models has hidden costs.

The first one is learning. Learning what it can do, how to prompt it, what its strengths and weaknesses are. That costs you time, and it makes the subscription the cheap part of the whole deal. You have to learn the tool, set it up in your workflow, and clean up all the big and small mistakes it makes. Just in time for another tool to come along that you want to try. Not to discourage you from attempting any of this, because if you can get it working, you’re going to save yourself a lot of time. It’s just a fact of any software.

That leads to the next point: something you pick today might not be the best thing in six months, or even three months from now. Companies keep updating models. They keep changing the price. They keep killing features. They keep adding modifications and safety nets to their models. You may build something that really works for you today, and in three months either something better comes up, or a really powerful feature no longer works because of some regulation or safety net they put on. That has nothing to do with you, but it does affect you.

Make sure you understand your processes. Meaning, if AI builds something for you that runs some automation, make sure you know at the very least the bare bones of it. It does this, then it does that. That way, if you move to a new model, you can rework it with knowledge instead of being lost on how AI built something, and you have a base to give the new model. “This is how this system worked,” or at least how the flow was supposed to go, instead of re-explaining your entire business use case. Documentation is really important. You can even have AI build the documentation. But I’d encourage you to make sure you know what the system is doing. In the end, it just means keeping your processes understandable, which is a good business practice overall.

Lastly, don’t lean on one tool. Like any other company, if you lean on one thing and that one thing disappears, you lose your entire business workflow. That’s a huge problem, especially if you start relying on it in place of people working. Another possibility is that you get into a dispute with the company that created the model, and you can’t use it anymore. Whatever it may be, you and I have both been through business problems like that before, and relying on one model and one system, at least right now with how new everything is, is probably a bad idea.

That’s the first one. In two weeks, I’ll bring you the 20-minute check that shows you who could lock you out of your own business, and what to fix today. If that sounds interesting, subscribe so we can send it to you.

If you’d rather talk about AI and how it fits in your business, that’s the kind of thing I like doing. You can grab some time with me here.