Disclosure: we make ThumbnailPeak, and this post uses its critique and Compare. Every score and verdict below is what the tools returned on 21 October 2026, on our own account. The video and its thumbnail belong to DIY PRO, who published them under a Creative Commons licence; the other four ideas are our adaptations.
YouTube's own A/B test takes up to three thumbnails. If one clearly earns more watch time, YouTube keeps it. If none does, YouTube keeps the first one you uploaded. (Here's how the test works.)
That gives you two decisions before you start. Which three go in, if you have more ideas than that? And which one goes first, since it's the one you're left with when the test can't decide?
We worked through both with five real ideas for one video. Here's what we found:
- One idea was broken. The critique scored it lowest, and Compare picked another idea over it with high confidence. It also had a mistake neither of them noticed.
- Two ideas were the same picture. One had words added, and it won clearly. A test with both would have spent a slot on a small difference.
- Of the three left, one beat the other two clearly, so it goes first. The other two were a coin toss, and which one won depended on whether Compare was given the title.
The video and the five ideas
The video is DIY PRO's "$20 Rust Bucket into Kick $$$ Go Kart W/ Harbor Freight 212": a rusty go-kart frame found for $20, rebuilt with a Harbor Freight 212 engine. It was published in 2024, but we treated it as a video about to go up, so all five ideas start equal.
The five ideas:
- DIY PRO's own thumbnail, the one the video was published with: the rusty frame in a small inset, a pink arrow, the finished green kart. No words.
- "RUST → GREEN", made by ThumbnailPeak's AI thumbnail generator from the video: the rusty frame beside the painted one.
- "212 POWER", from the same generator run: the engine beside the green frame.
- "2 SEATS", the third concept from that run: a seat, the frame and hands fitting a steering wheel.
- "$20 KART": DIY PRO's own thumbnail with the price added in big yellow letters, made with ThumbnailPeak's Edit by typing that one change.
We gave every tool the video's real title each time. The scores are ThumbnailPeak's critique, out of 100: seven checks on the picture and three on how well it fits the title.
Step 1: drop what's broken
2 SEATS scored 83, nine points below the next lowest. Its critique's first fix was "Establish a single focal point": the seat, the frame, the wheel and the hand "all compete for attention at feed size". We then compared it with RUST → GREEN, and Compare picked RUST → GREEN clearly, with high confidence, the surest verdict of the day.
There's a bigger problem neither tool caught: the headline says "2 SEATS", and the picture shows one seat. We noticed it ourselves in September, when the generator first made it. A viewer who clicks for a two-seater and sees one seat has been misled, and YouTube's test rewards watch time, not clicks.
So the first cut is easy, and it's one you can usually make without any tool: read every idea against the video. If it shows or says something the video doesn't deliver, it's out.
Step 2: don't spend two slots on one picture
Two of the ideas are the same photo: DIY PRO's own, and the same thumbnail with "$20 KART" on it. Testing both would tell you whether those two words help. That's a fair question, but it uses two of your three slots on one picture.
Compare answered it for us first. The "$20 KART" version won clearly, with medium confidence, 97 to 92. Compare's reason: "Its bold '$20 KART' headline remains readable at 168×94 and immediately communicates the cheap-build hook from the title." Three of the ten checks went up a grade: curiosity, promise consistency and title complementarity.
So we kept "$20 KART" and dropped the plain version.
Step 3: put the strongest first
Three ideas left: $20 KART (97), 212 POWER (93) and RUST → GREEN (92). Scores that close don't say much on their own, so we compared each pair.
- $20 KART beat 212 POWER clearly, with medium confidence. 212 POWER's "text is split into two lines and the kart is smaller and partly cropped."
- $20 KART beat RUST → GREEN clearly, with medium confidence. "'RUST → GREEN' also reads, but does not surface the $20 hook."
- 212 POWER beat RUST → GREEN by a hair, with low confidence, because it shows the 212 engine the title names. We then ran the same pair again without the title, and RUST → GREEN won by a hair instead.
So the order we'd upload is:
- A: $20 KART. It won both of its comparisons clearly. If the test ends without a winner, it's the one we'd want to keep.
- B: 212 POWER.
- C: RUST → GREEN.
B and C could swap without anything lost. Compare couldn't tell them apart with any confidence, and that's the useful part: it's exactly the kind of question YouTube's test is for. Real viewers will settle what an AI judge can't.
On a video that's already up, the current thumbnail should usually be A. Then, if the test can't decide, nothing changes. A new idea only replaces it by clearly earning more watch time.
Step 4: three last checks before the upload
- Every file at least 1280×720. If one option is smaller, YouTube shrinks all three to 854×480 for the test. (Size guide.)
- The bottom-right corner clear. YouTube puts the video's length there on every thumbnail. (Safe zone.)
- The three really are different. Ours are: the price, the engine, the before and after. Three versions of one idea would give YouTube little to choose between.
If you keep your thumbnails in a ThumbnailPeak project (a Pro feature), its A/B test section does most of this for you. Its default for our project was the same three, in the same order. Here's how the project export works.
What this cost, and the shortcut
We ran six comparisons for this post, and each comparison includes the critique of both thumbnails. A free ThumbnailPeak account includes 2 critiques and 1 comparison a month, so the full version is a job for Pro, which includes 50 comparisons a month.
You rarely need all six. The shortcut:
- Drop anything that misleads, by reading it against the video. Free.
- If two ideas are the same picture, compare them and keep the winner. One comparison.
- Compare your strongest remaining idea with the next one to decide what goes in A. One comparison.
What Compare can't tell you
Compare is an AI's judgement of two pictures and a title. It's good at spotting a broken idea, a muddled picture, or words that say more than the picture does. It can't see your audience, and it can't measure watch time. When it says "by a hair" with low confidence, believe it: that's a toss-up, and only YouTube's test can settle it.
That's also what we see across everyone's comparisons. Of the 1,696 comparisons run on ThumbnailPeak between 15 August and 21 October 2026, leaving out our own, 79% of those where the two thumbnails scored within 2 points ended in a slight win or a tie. Our three go-kart ideas, at 92, 92 and 93 before the words were added, were that close.
Frequently asked questions
How many thumbnails can I test on YouTube?
Up to three in one test. You can also test up to three titles, or three title-and-thumbnail pairs.
Which thumbnail should go first in a YouTube A/B test?
Your safest bet, because if no option clearly wins, YouTube keeps the first one you uploaded. On a video that's already published, that's usually the current thumbnail.
Should I test two versions of the same thumbnail?
Only if that one difference is the question you most want answered, and it's big enough to see at feed size, like adding words. Otherwise, use the slots for different ideas.
Can AI tell me which thumbnail will win a YouTube A/B test?
No. A tool like Compare can say which of two thumbnails is clearer and fits the title better, and how sure it is. Only YouTube's test measures what your viewers actually watch.
The five ideas were critiqued, and the pairs compared, on ThumbnailPeak on 21 October 2026, with the video's title and no description. The video is "$20 Rust Bucket into Kick $$$ Go Kart W/ Harbor Freight 212" by DIY PRO, published under CC BY 3.0. DIY PRO's own thumbnail is shown as published; the other four are our adaptations of its frames and thumbnail, and DIY PRO did not make or endorse them. The figures across all comparisons count those run on ThumbnailPeak's current comparison prompt, leaving out our own account's; we read only the scores and verdicts.
