Headline analyzers are a popular way to come up with great headlines and try to predict their effectiveness.
All you need to do is write your headline and the tool will give it a score telling you how effective it’s likely to be.
But can these free headline analyzers accurately predict which headline will perform best? Or are they yet another tool that keeps us busy and prevents us from focusing on what matters?
That’s the question this article will try to answer.
The 3 popular headline analyzers I tested:
In order to see whether headline analyzers could accurately predict how well a headline will perform, I took the results of 23 headline A/B tests I ran on my blog French Together and answered a simple question:
Is the highest-performing headline according to the A/B test also the headline with the highest score?
There are lots of headline analyzers I could have used for the test but I decided to focus on the 3 most popular ones since they are the ones you are most likely to use.
For the A/B test, I chose Thrive Headline Optimizer, a popular Wordpress plugin.
Before showing you the results of the test, let’s take a look at the 3 headline analyzers I tested!
CoSchedule Headline Analyzer
CoSchedule headline analyzer looks at word balance (common, uncommon words, power words, emotional words…), keywords and overall sentiment (positive, negative).
It then gives the headline a score, offers tips to improve it, and gives you useful information such as whether your headline is too long or too short.
ShareThrough Headline Analyzer
ShareThrough headline analyzer looks for context words based on a study showing that readers are likely to be more interested in your message if you use one (or several) of 1,072 of the context words that exist in the English language.
The idea is that using context words like “everyday”, “considering” or “secrets” will make your headlines more noticeable and drive more shares and traffic to your articles.
In addition to context words, the blog title analyzer also looks for positive and negative sentiments and passive words like “guess”, “almost” or “doubt” and offers tips to improve your headline (mention a celebrity, increase headline length..)
Advanced Marketing Institute EMV headline analyzer
The Emotional Marketing Value Headline Analyzer looks for emotional words in your headline and gives it a score showing how emotional it is.
This is based on research from the 70s and on the idea that more emotional headlines perform better.
The test: can headline analyzers accurately predict winning headlines?
I compared A/B test results to the scores given by headline optimizers to see how accurate tools like CoSchedule headline optimizer are.
For each headline, I compared the score of the control to the score of the winning variation in order to see whether headline analyzers were able to correctly predict which headline will perform better.
Here are the headlines I tested and the scores they got in CoSchedule headline analyzer, Emotional Marketing Value headline analyzer and ShareThrough headline analyzer.
Headline #1
Control: The Beginner’s Guide to Mastering the French Present Tense
Winner: How to Master the French Present Tense Once and for All +8.59%
Control CoSchedule score: 57
Winning variation CoSchedule score: 72
Control EMV score: 22.22%
Winner EMV score: 18.18%
Control ShareThrough: 77
Winner ShareThrough score: 77
Headline #2
Control: The 14 French TV Shows You Should Watch Next
Winner: From Horror to Laughter: The 14 Tv Shows Every French Learner Should Watch +5.32%
Control CoSchedule score: 65
Winning variation CoSchedule score: 56
Control EMV score: 33.33%
Winner EMV score: 38.46%
Control ShareThrough score: 78
Winner ShareThrough score: 72
Headline #3
Control: What Do the French Eat for Breakfast? Not What Most People Expect
Winner: What Is a Typical French Breakfast like – and Why? +4.94%
Control CoSchedule score: 66
Winning CoSchedule score: 71
Control EMV score: 33.33%
Winner EMV score: 0.00%
Control ShareThrough score: 74
Winner ShareThrough score: 65
Headline #4
Control: 41 French Youtubers to Follow to Finally Understand Spoken French
Winner: 41 Popular French Youtubers That Will Help You Better Understand Spoken French +19.05%
Control CoSchedule score: 52
Winner CoSchedule score: 66
Control EMV score: 40.00%
Winner EMV score: 16.67%
Control ShareThrough score: 70
Winner ShareThrough score: 71
Headline #5
Control: The 14 Best Youtube Channels to Learn French
Winner: The 14 Best Youtube Channels to Learn French for Free (Ranked by Level) +6.13%
Control CoSchedule score: 64
Winner CoSchedule score: 60
Control EMV score: 12.50%
Winner EMV score: 15.38%
Control ShareThrough score: 68
Winner ShareThrough score: 70
Headline #6
Control: 23 Must-See French Movies
Winner: The 23 Best French Movies for French Learners to Watch in 2019 +14.82%
Control CoSchedule score: 42
Winner CoSchedule score: 63
Control EMV score: 50%
Winner EMV score: 33.33%
Control ShareThrough score: 65
Winner ShareThrough score: 63
Headline #7
Control: The Beginner’s Guide to the French Imparfait (How and When to Use It)
Winner: How and When Should You Use the French Imparfait? + 9.64%
Control CoSchedule score: 60
Winner CoSchedule score: 70
Control EMV score: 23.08%
Winner EMV score: 22.22%
Control ShareThrough score: 74
Winner ShareThrough score: 72
Headline #8
Control: 18 Delicious Food-Inspired French Idioms
Winner: The 18 Funniest French Sayings About Food + 15.48%
Control CoSchedule score: 48
Winner CoSchedule score: 63
Control EMV score: 20.00%
Winner EMV score: 28.57%
Control ShareThrough score: 67
Winner ShareThrough score: 73
Headline #9
Control: How to Find Slow French and Turn Any Audio into Slow French
Winner: Is Listening to Slow French Bad for You? (And How to Do It Anyway) +17.35%
Control CoSchedule score: 59
Winner CoSchedule score: 69
Control EMV score: 41.67%
Winner EMV score: 35.71%
Control ShareThrough score: 74
Winner ShareThrough score: 78
Headline #10
Control: Why You Shouldn’t Study Grammar Rules and What to Do Instead
Winner: Don’t Study French Grammar, Do This Instead! +14%
Control CoSchedule score: 59
Winner CoSchedule score: 59
Control EMV score: 45.45%
Winner EMV score: 28.57%
Control ShareThrough score: 68
Winner ShareThrough score: 65
Headline #11
Control: Why Saying “How Are You” in French Isn’t as Easy as You Think
Winner: How to Say “How Are You” in French and Sound like a Local +9.88%
Control CoSchedule score: 68
Winner CoSchedule score: 72
Control EMV score: 30.77%
Winner EMV score: 23.08%
Control ShareThrough score: 71
Winner ShareThrough score: 74
Headline #12
Control: Bonne Nuit and Other Phrases You Can Use to Say Good Night in French
Winner: 31 Words French People Use to Say Good Night and Talk About Sleep +12.59%
Control CoSchedule score: 51
Winner CoSchedule score: 59
Control EMV score: 28.57%
Winner EMV score:30.77%
Control ShareThrough score: 75
Winner ShareThrough score: 81
Headline #13
Control: 21 Shades of Oui: How Locals Say Yes in French
Winner: 21 Ways to Say Yes in French Without Sounding Like a Tourist +12.7%
Control CoSchedule score: 61
Winner CoSchedule score: 64
Control EMV score: 30.00%
Winner EMV score:8.33%
Control ShareThrough score: 72
Winner ShareThrough score: 74
Headline #14
Control: Do You Know the 16 Phrases the French Use to Say “Congratulations”?
Winner: 16 Ways to Say Congratulations in French (And Sound like a Local + 7.86%
Control CoSchedule score: 63
Winner CoSchedule score: 67
Control EMV score: 33.33%
Winner EMV score:8.33%
Control ShareThrough score: 74
Winner ShareThrough score: 74
Headline #15
Control: 10 Things You Need to Know Before You Wish Someone a Happy Birthday in French
Winner: What you need to know before you wish someone a happy birthday in French + 8%
Control CoSchedule score: 74
Winner CoSchedule score: 74
Control EMV score: 40.00%
Winner EMV score:42.86%
Control ShareThrough score: 75
Winner ShareThrough score: 72
Headline #16
Control: Putain: The Ultimate 2019 Guide to the French’s Favorite Curse Word
Winner; Why Do the French Say Putain so Much? (And How to Use It Right) + 18.48%
Control CoSchedule score: 67
Winner CoSchedule score:70
Control EMV score: 9.09%
Winner EMV score:21.43%
Control ShareThrough score: 65
Winner ShareThrough score: 77
Headline #17
Control: Are You Unknowingly Insulting People by Using the Wrong “You” in French?
Winner: “You” in French: How to Choose Between Tu and Vous + 13.09%
Control CoSchedule score: 66
Winner CoSchedule score: 69
Control EMV score: 50.00%
Winner EMV score: 50.00%
Control ShareThrough score: 75
Winner ShareThrough score: 69
Headline #18
Control: Here Are the 8 Best French Dictionary Apps
Winner: 8 French Dictionary Apps You Should Use in 2019 + 10.77%
Control CoSchedule score: 75
Winner CoSchedule score: 61
Control EMV score: 25.00%
Winner EMV score: 33.33%
Control ShareThrough score: 67
Winner ShareThrough score: 62
Headline #19
Control: How to Use the Careful Rule to Know When to Pronounce the Final Letter of French Words
Winner: Should You Pronounce the Final Letter of French Words? (control) + 7.54%
Control CoSchedule score: 52
Winner CoSchedule score: 62
Control EMV score: 29.41%
Winner EMV score:11.11%
Control ShareThrough score: 78
Winner ShareThrough score: 75
Headline #20
Control: 7 French Pronunciation Tips That Will Help You Sound More French
Winner: 7 French Pronunciation Tips to Avoid Sounding Like a Tourist + 6.1%
Control CoSchedule score: 67
Winner CoSchedule score: 62
Control EMV score: 18.18%
Winner EMV score:20.00%
Control ShareThrough score: 74
Winner ShareThrough score: 70
Headline #21
Contrl:How to Say Hello in French (And Avoid Embarrassing Greeting Faux-Pas)
Winner: Bonjour and beyond: How to say hello in French + 15.73%
Control CoSchedule score: 66
Winner CoSchedule score: 68
Control EMV score: 18.18%
Winner EMV score: 33.33%
Control ShareThrough score: 75
Winner ShareThrough score: 72
Headline #22
Control: 64 Catchy French Songs Every French Learner Should Know
Winner: 64 Songs to Learn French with Music + 19.69%
Control CoSchedule score: 63
Winner CoSchedule score: 60
Control EMV score: 33.33%
Winner EMV score: 28.57%
Control ShareThrough score: 70
Winner ShareThrough score: 67
Headline #23
Control: The Beginner’s Guide to French Numbers (With Audio)
Winner: The Ultimate Guide to French Numbers (With Audio) + 21.97%
Control CoSchedule score: 60
Winner CoSchedule score: 64
Control EMV score: 37.50%
Winner EMV score: 37.50%
Control ShareThrough score: 68
Winner ShareThrough score: 67
Conclusion
As you can see, CoSchedule correctly predicted 16 winners out of 23 while the Emotional Marketing Value Headline Analyzer and ShareThrough Headline Analyzer only predicted 9 and 8 winners out of 23 respectively.
This makes CoSchedule headline analyzer the most accurate headline analyzer of the 3 with an accuracy of 69%.
This also means that headline analyzers really aren’t that reliable and that using them to choose a headline is probably not a good idea. A conclusion that is further confirmed by Mathias from the Journey of.
Instead, I recommend running a good old-fashioned A/B test. It will be slower but you will get more reliable results.
Do you use headline analyzers? Do you find them useful?
