I Tested Opus 5.5 at Every Effort Level. What You Need to Know.
핵심 요약
Opus 5.5의 성능을 검증하기 위해 동일한 3D 가상 컨퍼런스 월드 구축 프롬프트를 저(Low) 단계부터 울트라 코드(Ultra Code)까지 모든 노력 수준(Effort Level)에서 실행해 품질과 비용, 소요 시간 등을 비교 분석했다. 105GB의 실제 온라인 행사 녹화본을 바탕으로 3인칭 탐색 환경을 구현하도록 요청했으며, 가장 낮은 단계(Low Effort)에서는 기본적인 맵과 일정 구조는 잡았으나 사람 그래픽이 사라지거나 영상이 정지 이미지로 나오는 등 불안정한 완성도를 보였다.
주요 포인트
- **전 레벨 성능 비교 실험**: Opus 5.5에 동일한 프롬프트를 입력해 Low부터 Ultra Code까지 각 노력 수준별 결과물 품질, 실행 시간, API 비용, 토큰 수, 질문 횟수 등을 다각도로 비교함.
- **고난도 3D 월드 생성 과제 부여**: 105GB 분량의 온라인 컨퍼런스(AIS Live) 녹화 에셋을 기반으로, 3인칭 탐색이 가능한 사실적인 3D 가상 컨퍼런스 공간을 제작하도록 명령함.
- **Low 단계의 구조 파악 능력**: 최저 노력 수준에서도 행사 아젠다, 부스, 메인 무대, VIP 라운지, 실시간 맵 등 기본 골격과 행사 정보는 정확히 인식해 배치함.
- **Low 단계의 디테일 한계**: 브랜드 로고 및 고유 색상 미반영, 지나치게 밝은 조명, NPC 캐릭터 증발 버그, 영상 대신 정지 이미지가 슬라이드처럼 표시되는 등 렌더링 및 물리 완성도에서 뚜렷한 한계를 보임.
Opus 5.5, Opus 5.5, Opus 5.5, Opus 5.5, Opus 5.5. This model is literally everywhere and for very good reason. It's smart. It's cheap. It has amazing taste. It is an incredible AI model. But with every AI model, you have a choice of effort. Whether that be low, medium, high, extra, max, or ultra code. So in this video, I gave Opus 5.5 the same exact prompt, and I ran it on every single effort level. And we're going to be comparing the results. We will look at the quality of all of the different actual outputs, but then we're also going to look at how long each one of these ran, how much it cost us if it was API billing, the total tokens, how many checks they ran, and how many questions they actually asked me throughout the process. And the results that we got are not what I expected at all. So, I can't wait to share this with you guys. Let's not waste any time and just get straight into this one. Okay, so let's jump right into this one. I want to start off just by showing you guys the actual prompt that we used that we gave to every single one of these different agents. I'm going to go into the files over here and we're going to open up this prompt markdown file and I'll show you what we got. So, here is the / goal that I fed in. I said, "You need to create me a 3D world that is a realistic tech conference that I can walk around in thirdp person view. You will look at this folder, which is my event recording assets from AIS Live." In this folder is a frame IO folder of 105 GB of video recordings. This was a completely virtual event. Everything was recorded and all of the recordings are right here. I said, "Your goal is to take this event and turn it into a 3D explorable world that makes me feel like I actually went to a real in-person conference with different rooms, different tracks, different stages, blah blah blah. Feel free to use key.ai if you need to generate any images or videos. And you can also use anything else inside of my Herk 2 project, which is like my AI operating system." I said, "You'll be judged on creativity, design, physics, and the overall feel as I explore the 3D world that you built." And that was basically the end of the instructions. So, as you can see on this lefth hand side, I ran this through all of the different effort levels. Let's start off with low and work our way up to ultra code. All right, so here we have the result from low. Let's open this up and take a look. So we have AIS Live AI services summit in person finally and we are able to click around. First of all, this doesn't feel very branded. Like this isn't the AIS Live logo. It's not even our colors. So I don't love that. But let's get in here. Okay, this is way too bright. Um, we have a map in the top right. We have a city back here. I can't tell what city this is. Okay, this is Chicago, which is kind of cool because, you know, I live in Chicago. But anyways, on the top right we can see a map. We have a lobby. We have an expo hall. We have VIP lounge. Main stage. Also, the map shows where every other person is. And that's syncing live. So, we can see registration. We can see day one. Hyper agent keynote live debrief. Cool. So, it actually does know the agenda. And then there's day two. So, it found that. That's good. We have these little balls in here that I can hopefully kick around. Okay. The fix. Oh, look at this. If I go over here, all the people just disappear. Very bad. Very bad. Okay. So, let's see. Can I sprint? Okay, I can move a little faster. I'll first go over here. There's some swag. Um, AI+ certified glido. Okay, so it's got the actual booths that we had in the virtual event. We had booths. So, that's pretty cool. Little place to take pictures. Room C. Right now, we have Tangy Frederick doing a workshop. Okay, but this isn't a video. As you can see, it's just an image. She's not moving. So, that's just an image. These people are disappearing. They must be ghosts. Let's go over here to room A. We have Liberty White. Okay, very cool. Your first 30 days in automation. Once again, it's just a still image and the people are glitchy, so not very good here. I'm going to go over to the main stage and see what we've got going. Okay, cool. So, we have a main looking stage. The people are glitching really bad. That's not very good at all. Our video is actually moving. Like, I saw my face here and I saw Devon's, but now they're gone. So, I'm not sure what happened. Okay, it's it looks like it's more of a slideshow. Nothing is actually being played still. Anyways, let's come in here. We have more booths. We have Hyper Agent. We have Claude Code. We have more swag. Room B's Dave Evelar. I'm assuming it's the same exact thing. We've got coffee. And then I'm assuming VIP lounge, VIP access only. That's pretty cool. But there's really nothing going on right here. That screen's way too bright. Okay, so I think you guys understand the vibe that we're getting here from Opus 5.5 on low effort. And here's where things get in