Hello!
We are seven emails in and they have all been about what AI can do for you. This one is about what it can't do, because I've been getting way more clear-eyed about it recently and I'd definitely be doing you a disservice if I only sent you the good news.
I'll give you a couple of examples that have happened to me more than once. I ask it something and it gives me a confident, well-written, completely convincing answer - but it's wrong.
Not even slightly wrong. It has made up supplier terms that don't exist, got numbers and calculations out by a factor that mattered and told me pieces of employment law that turned out to be from a different country. Those are just the three that come to mind first. Every one of these times it sounded just as sure as it does when it is correct, which is my whole problem with it. It just doesn't know when it doesn't know!
The other thing I want to point out and this one took me longer to see clearly, is that it is an out-and-out yes man! AI models have been built to keep you happy, which in turn keeps you using the app which in turn grows their profit. So it works out what you want to hear and what makes you feel good and it learns that way.
If you ask it if your idea is good, it will usually find a way to say yes. If you ask it to review something you wrote, it'll tell you it's great before it tells you anything useful. Don't mistake this for being nice. This is the product doing its job and you really have to fight it!
We have sort of turned off the flattery switch by putting instructions behind our AI models to disagree with us, point out the weak bits first and tell us when it's not sure rather than making something up. It's one of the things we've done to get round it and it makes a noticeable difference.
Here's a few rules I always suggest to people.
Use it freely for the low-risk stuff:
Explaining a contract clause
Working out what a fault code means
Drafting something for you
Summarising a document or a book
Even asking it daft questions you'd be embarrassed to ask a person which lets be honest - we all do!
Be careful with anything it counts - so numbers, sums, dates or anything that has to be exact. We keep the maths out of AI entirely. We do the maths with formulas, rules and automation, which give the same answer every time and only then does the AI get to look at the results. Our AI guy was firm on this from day one, and he was right.
Don't base important decisions on its output alone. Things like legal, tax, health and safety, anything with a contract or a lot of money attached. Use it to understand where you stand and what to ask, then take that to the person who is actually accountable for it. You'll get more out of the professional, and you'll probably pay them less in the long run.
Overall, make sure you read everything before it leaves the building: quotes, emails, minutes, absolutely everything. I've said that in most notes and I'll keep saying it. The best way to get a feel for where it works versus where it's guessing is to ask it how sure it is, or ask where it got the information from and then go and check yourself now and again. You'll be surprised how often the answer changes when you push it and challenge it.
None of this changes anything from the earlier notes. The setups I've sent you will still save you hours and add real value. Just don't stop checking because it's been right a lot recently. That's usually when it catches you out.
Feel free to reply and tell me the worst thing it's confidently told you. I'll use the best examples in a future email, names removed of course.
Andrew
Ps. If anyone's selling you AI as something that means you don't need to think anymore, walk away. And please, don’t stop thinking for yourself