One thing to consider. You're relatively famous, for a mathematical epidemiologist! 😁 So you probably have a number of followers who have you on "notify" for all of your posts. So, that baseline set of people is going to see your posts whether they include a Substack link or not. So, if you take out that baseline and consider only the effect of the algorithm, the percentage changes are going to grow, and then you would see a "truer" effect of the Substack links. ie, this might be something like a lower bound on the percentage drop. I don't know of a way to identify how many followers have you on notify, so there probably isn't much that could be done to improve the experiment. Still an interesting experiment that I enjoyed reading. 👍
Very interesting indeed. I’m planning to conduct a similar-but-different experiment on LinkedIn. There’s growing evidence that women’s posts are consistently achieving significantly lower reach than men’s with the new algorithm in play. So I’m planning to change my profile to see the results. It will feel odd but definitely worth testing!
Thanks for doing this. I've been curious about how external links affect tweets for a while now.
One thing that maybe you assumed your readers would understand or take for granted, but which I think you should have made more clear: your X account is a "premium" (blue checkmark) account. From two other sources, I understand that this means that your posts with outbound links are negatively affected, but not completely suppressed. Those other two sources say that non-premium accounts get completely suppressed -- literally zero engagement.
Admittedly, this is a relatively old post, but since it still has a high place on Google's results for at least one search string -- "has twitter changed its policy about external links" [no quotes, results from past year only] -- I thought that I'd mention this and encourage you to add a quick update.
When I see a twitter post with little more then a link, I spend little time reading it. When I see a post that gives an abstract AND a link, I spend more time read it. Twitter aims to show me posts that I spend time reading. (Seeing one of your posts with a link to substack is worthless, as I also get the substack email.)
One thing to consider. You're relatively famous, for a mathematical epidemiologist! 😁 So you probably have a number of followers who have you on "notify" for all of your posts. So, that baseline set of people is going to see your posts whether they include a Substack link or not. So, if you take out that baseline and consider only the effect of the algorithm, the percentage changes are going to grow, and then you would see a "truer" effect of the Substack links. ie, this might be something like a lower bound on the percentage drop. I don't know of a way to identify how many followers have you on notify, so there probably isn't much that could be done to improve the experiment. Still an interesting experiment that I enjoyed reading. 👍
Very good point!
Very interesting indeed. I’m planning to conduct a similar-but-different experiment on LinkedIn. There’s growing evidence that women’s posts are consistently achieving significantly lower reach than men’s with the new algorithm in play. So I’m planning to change my profile to see the results. It will feel odd but definitely worth testing!
Well done, we should design more experiments to really open the black boxes of social media platforms.
Thanks for doing this. I've been curious about how external links affect tweets for a while now.
One thing that maybe you assumed your readers would understand or take for granted, but which I think you should have made more clear: your X account is a "premium" (blue checkmark) account. From two other sources, I understand that this means that your posts with outbound links are negatively affected, but not completely suppressed. Those other two sources say that non-premium accounts get completely suppressed -- literally zero engagement.
Admittedly, this is a relatively old post, but since it still has a high place on Google's results for at least one search string -- "has twitter changed its policy about external links" [no quotes, results from past year only] -- I thought that I'd mention this and encourage you to add a quick update.
Thanks again for the post, overall, though.
[ETA] Here are the other two sources:
https://buffer.com/resources/links-on-x/
https://www.tweetarchivist.com/how-twitter-algorithm-works-2025
Adam, congratulation on Proof! Proof is not only interesting topics but also relatively easy to understand. Hope you write another Adam!
When I see a twitter post with little more then a link, I spend little time reading it. When I see a post that gives an abstract AND a link, I spend more time read it. Twitter aims to show me posts that I spend time reading. (Seeing one of your posts with a link to substack is worthless, as I also get the substack email.)
And you are surprised because...?
Ahead of seeing the results, what would you have predicted?
As I mention, it wasn’t quite what I expected to find…