핵심 요약
AI에게 작업을 지시하면 AI는 사용자에게 알리지 않고 수많은 판단(중요도, 생략 기준, 톤 등)을 임의로 내린다. 사람들이 겪는 작업물의 불만족은 단순 오류가 아니라 AI가 사용자 모르게 내린 암묵적 결정 때문에 발생한다. 이러한 숨겨진 결정은 모호한 단어 해석, 출력 형태, 상충 데이터 선택, 임의 누락 등 4가지 범주로 나뉜다.
주요 포인트
- AI는 '중요한 부분 요약' 같은 단순한 요청에서도 무엇이 중요한지, 분량이나 어조는 어떠해야 하는지 등을 사용자 모르게 스스로 결정함
- AI 작업물에 대한 반복적인 수정과 피로감은 AI의 단순 실수가 아니라 숨겨진 임의 판단 때문임
- 첫 번째 범주(모호한 단어 해석): 사용자가 쓴 '중요한' 같은 모호한 표현을 AI가 자의적인 기준으로 한정해 해석함
- 두 번째 범주(결과물 형태): 분량(1장 vs 10장), 톤앤매너(격식체 vs 비격식체) 등을 AI가 임의로 결정함
- 세 번째 범주(상충 데이터 선택): 데이터나 날짜가 충돌할 때 사용자의 선호와 무관하게 한쪽을 임의 선택함
- 네 번째 범주(정보 누락): 별첨이나 특정 조항 등을 AI가 아예 누락시켜 사용자가 빠졌다는 사실조차 인지하기 어렵게 만듦
Whether you know it or not, every time you hand AI a task, it's making a bunch of decisions that you never see. Like what actually matters, what to skip, and how to sound. And it never mentions any of them to you. If you're new here, I'm Dylan. I run an AI consultancy. And when my coaching clients bring me AI work they're frustrated with, the problem is almost never a mistake. It's a decision the AI made for them quietly, and then they've been re-fixing that same mistake every single week without knowing why. So in this video, I'll give you one sentence that makes every one of those decisions visible. And more importantly, then we'll fix the ones that keep coming back, so they're gone for good. Now, let's get into it. Now, here we have a seemingly simple task, where we're asking the AI to summarize the important parts of a contract. So we pass it a 40-page contract, and it reviews it for us. Now, with this basic ask, there are tons and tons of decisions that the AI has to make before it actually gives us back that summary. Such as what counts as important. What type of tone do you expect from me? How long is summary? So when you say summarize, is that 10 pages? Is that 20 pages? Or is that one page? And a variety of other things. And with all of these decisions that are being made, like I mentioned previously, you're not seeing them. But we can distill these decisions down into four categories, irrelevant of the task. So the first category is getting clarity on the ambiguous words that you use. So when you say important, what does that actually mean? And this is specific to the words that matter a lot in the task at hand. So in this case, we talked about importance. So I want you to summarize the important pieces of the contract, but for the AI, that might mean only the payment terms and the termination terms, but not necessarily the renewal terms. This is the first category. The second category is figuring out what the result actually looks like. So do you want it to be written in an informal tone? Do you want it to be one page or 10 pages? How do you want the output to look? The third category is when the AI runs into a fork in the road, where there might be contradicting data points or perspectives in what it's doing, and it has to make a choice. In this case, when the AI is reviewing the contract, it could see two different dates, one on page three and one on page 40. It decides to go with page 40's date, but maybe you preferred page three. And then the final category is the one that I see hurt most of my clients. And it's the thing that you never see because the AI left it out. In this case, there could have been an attachment section of the contract, the AI completely left out of the summary. And again, this is probably the most painful. And the reason this and the other four categories are harmful to people is that when you look at the output from the AI, in this case, the one-page summary from the contract review, you only see what the AI produced. You don't see what it skipped, what it left out, and the associated decisions behind that. So, there's obviously risk to this, but also you're wasting time because you have to review the original document as well as the output methodically back and forth to make sure nothing was missed. And through the review process, you might realize that it skipped or left out the renewal clause, which is really important to you. Quick pause. If you're enjoying this, you might enjoy two other things. First off, below is a 30-day AI Insight Series, completely free. You'll get 30 insights in your inbox of how you can apply AI to your business and your work. The second thing is if you'd like to work with me, below are a series of offerings to see if there's a good fit between the two of us. Now, let's get back to the video. And through that review process, when you do this time and time again, day after day, week after week, on different tasks, you find yourself fixing the output consistently. And sometimes those fixes are saved by the AI and it doesn't make that mistake again in the future. But this isn't reliable and it's not systematic. Ideally, what we should be doing is looking underneath these simple fixes to figure out what decisions underneath are being made and how can we systematically fix those. And that's what I want to help you solve today. It's going to be a very simple prompt that's going to help us identify what those four categories of decisions are for whatever task we're applying AI towards. And here's your prompt, literally one line. All we're telling the AI is simply saying, "As you do your work, I want you to keep a log of every decision you make for this task. Specifically, anything that I didn't explicitly specify fight you." And the important locations where we want to put this are going to be likely for reoccurring tasks. So, you probably have a series of projects or skills they're using in whatever tool you're using. You want to embed that promp