> ## Content Index > Fetch the complete content index at: https://chrisenns.com/llms.txt > Use this file to discover other available public pages before exploring further. # YouTube Testing Everything - URL: https://chrisenns.com/2026/10/youtube-testing-everything/ - Published: 2026-10-07T16:06:47.000Z - Updated: 2026-10-07T16:06:47.000Z - Description: Does A/B testing suck the creativity out of a video? - Author: Chris Enns - Tags: YouTube, creativity Via Marcin Wichary's excellent blog, [Unsung](https://unsung.aresluna.org/have-conviction-in-what-you-want-to-share-with-the-world?ref=chrisenns.com), where he riffs off Marques Brownlee (aka MKBHD)'s [video talking about YouTube's new "feature"](https://www.youtube.com/watch?v=e1q-TuHdc4Y&ref=chrisenns.com) of letting creators A/B test their videos: > On top of all that, we can add what Brownlee started with in his video: overeager A:B tests feel like software’s version of gaslighting. How can you trust an app if it feels like it’s changing without rhyme and reason seemingly on a weekly basis, whose changes are not announced or documented anywhere (small A:B tests never are), and when other people don’t see the things you see? I'm only adjacent to software A/B testing conversations [via my work](https://www.lemonproductions.ca/?ref=chrisenns.com), but the conversations I hear and read among software developers feel like it makes sense to use some A/B testing in that area to see what's working and what's not. Which is why A/B testing the title or thumbnail feels somewhat useful if you're making slight changes to each and see which one people click through on more often. And even if we set aside the artistic vision of creating a video and having your own opinion on what works, A/B testing a video based purely off someone watching something on [my YouTube channel](https://www.youtube.com/@lemonpodcasting/?ref=chrisenns.com), that feels really subjective and random. Wichary quotes [Jim Nielsen's post](https://blog.jim-nielsen.com/2026/dear-software-makers/?ref=chrisenns.com) where he said: > I find it interesting how Brownlee shares his opinion that YouTube spends way too much time chasing their competitors and not enough time being YouTube. Amen. I wish YouTube believed in itself as *the* platform with as much confidence as my kids talk about only needing YouTube in our home over any other streaming platforms. One counterpoint to the discussion that I haven't seen mentioned yet is how much Real Hollywood Directors™ screen test their movies before they put them out. According to [Wikipedia](https://en.wikipedia.org/wiki/Test%5Fscreening?ref=chrisenns.com): > Preview audiences are selected from a cross-section of the population and are usually asked to complete a [questionnaire](https://en.wikipedia.org/wiki/Questionnaire?ref=chrisenns.com) or provide feedback in some form Lots of interesting anecdotes in that Wikipedia page about movies changed based on test screenings, which aren't exactly an A/B test with the same audience but are a variation on that of testing and see if what you've changed performs better: > According to a June 2008 article from *The Guardian*, "Two weeks before the release of [*The Bourne Supremacy*](https://en.wikipedia.org/wiki/The%5FBourne%5FSupremacy%5F%28film%29?ref=chrisenns.com) (2004), director [Paul Greengrass](https://en.wikipedia.org/wiki/Paul%5FGreengrass?ref=chrisenns.com) got together with its star, [Matt Damon](https://en.wikipedia.org/wiki/Matt%5FDamon?ref=chrisenns.com), came up with a new ending and phoned the producers saying the new idea was "way" better, it would cost $200,000 and involve pulling Damon from the set of [*Ocean's 12*](https://en.wikipedia.org/wiki/Ocean's%5FTwelve?ref=chrisenns.com) for a [re-shoot](https://en.wikipedia.org/wiki/Re-shoot?ref=chrisenns.com). Reluctantly the producers agreed—the movie tested 10 points higher with the new ending". And we wouldn't have the POW and BANG effect in the 1960's Batman series if it wasn't for screen testing: > [Adam West](https://en.wikipedia.org/wiki/Adam%5FWest?ref=chrisenns.com), in his book *Back to the Batcave*, stated that test screenings for the 1960s [*Batman*](https://en.wikipedia.org/wiki/Batman%5F%28TV%5Fseries%29?ref=chrisenns.com) television series incorporated audience-controlled dials monitored by computer. Shown to about one hundred recruited audience members, the pilot episode received "the worst score in the history of pilot testing", in the "high 40s", where the average pilot score was in the mid-60s. Several adjustments were made to the show and retested, including a [laugh track](https://en.wikipedia.org/wiki/Laugh%5Ftrack?ref=chrisenns.com), then [narration](https://en.wikipedia.org/wiki/Narration?ref=chrisenns.com); the test results were the same. The decision was made to add "huge new special effects gags that would look great in promos. It's not really fair—or maybe even helpful—to compare a major movie production to a YouTube video, but it does feel worth pointing out that testing your video with an audience can be helpful to the creative process and isn't always about how many clicks or dollars can we extract from this piece of content. As with Shaun of the Dead, it can sometimes make it better: > [Edgar Wright](https://en.wikipedia.org/wiki/Edgar%5FWright?ref=chrisenns.com), writer and director of [*Shaun of the Dead*](https://en.wikipedia.org/wiki/Shaun%5Fof%5Fthe%5FDead?ref=chrisenns.com), said in an interview that in test screenings done before the film's special effects were completed, audiences remarked that the ending was "a bit abrupt" and "lame". After being given a low budget and two days to finish shooting, the filmmakers added a "15 second" ending, which follow-up press screening audiences liked, leading to one reviewer changing his earlier bad review, giving "an extra star". You can do this with YouTube right now if you wanted by emailing a select group of people an unlisted video on a channel. The video isn't being pushed out yet, you get feedback from viewers, and could edit and upload a new video before making it public. I honestly don't know how many YouTube creators would be doing a manual test screening of their videos in a similar style to a Hollywood studio, but I have to imagine it's happening.