How to get around the WSJ paywall
The Wall Street Journal is famous for its “paywall” requiring you to subscribe to read content online. Unlike some other notable competitors (namely the New York Times), the WSJ allows 0 articles a month to be read without a subscription. I pay for the NYT and some others, but I can’t bring myself to give money to Murdoch or anyone on their crazy editorial board.
The interesting part about the WSJ paywall is that you can read every article for free, but only if you are coming to each article from Google. If you click a link from an email it will try and make you pay, if you click the same link from a Google search you can read the full article. Because of this my general routine when getting passed a WSJ link was:
- Visit, seeing the first sentence or two and the headline along with a giant Subscribe ad
- Copy the headline and paste into Chrome’s address bar, which searches Google for the headline
- Click the first result which would inevitably be the WSJ article in question
- Read the article in full
This technique works well, but is an annoying process to go through. I recently decided to automate this and after some failed testing with using automated Google searches and the “I’m feeling lucky” button, I found that the only thing that mattered was making the initial request with a referrer of www.google.com.
There’s a wonderful Chrome extension called Referer Control that lets you control referrers (crazy right?). I created a rule matching http://online.wsj.com/* that sends the referrer https://www.google.com. Low and behold, no more paywall! I can click a link to a WSJ email and immediately see the full text.
I don’t use Firefox much anymore, but there’s a similar extension available for it that should do the same trick.
tl;dr
When visiting the WSJ, set your referrer to be from Google and the paywall disappears. If you use Chrome or Firefox there is a browser extension that makes this easy.
Update
Sometimes you can come across a WSJ link that is to www.wsj.com instead of online.wsj.com, all it takes to make sure this technique works then is to create another rule for that base address. So http://www.wsj.com/* and you can keep the same referrer.
Thanks for sharing this!
Please help me set this up for Firefox using Referrer Control brah
Fascinating, this still works.
(I wonder if it violates Google’s search-system terms of service for a web site to provide different data to the robot than to a normal human.)
Wow. Just, wow. Awesome.
Fantastic method, thank you for the solution!
I’m going to have to give this a try because I’m a cheapskate too. Had a chuckle at the obligatory progressive disclaimer. IOW: “Murdoch et. al. are loonies, but I want to read their content so badly I’m willing to twist myself into a pretzel to get at it.”
Lovely!
No worky… :-/
I binged on BOTW columns via the manual way (search headline in Google, click the result) and apparently there’s a limit for this. Now I only get the “subscribe” ad. Same through Referer Control. And also regardless of browser or device… Any way around that? Flushing cookies etc. doesn’t seem to help.
And while I’m a cheapskate, I would even pay for access, but not for the whole WSJ site…
I’d be really surprised if cookies don’t do the trick. You may need to dump Flash cookies though, those usually stay around. An easier way to do that is to not enable Flash… 😉
Hi JG
Downloaded the “Referer Control” extensions and the suggested settings for WSJ but it no longer works.
Any idea if it still works for you?
Thanks!
Yes, it works for me. You may need to delete your cookies first. There could be a limit to how many times this works (e.g. it lets you view 10 articles a month), deleting your cookies will be able to prevent that. Turning off Flash can help too as “Flash cookies” can be tricky to remove. I have Flash set as “click to play” in Chrome which means I have to allow Flash to run before it executes, no hidden stuff in the background.
Think this method seems to be now blocked =/
It continues to work for me, maybe you need to delete cookies?
What cookies do I need to delete?
Seems it only works for the articles that are not tagged by key. Probably these articles have a hard paywall.
Was using refcontrol and cookie manager +, was working fine till about 2 weeks ago. Then could not get articles at wsj nor if I went to google. Blocked adobe cookies and could get articles through google but still not on site. Any info would be appreciated.
This still works clear cookies.
wow. Brilliant. worked. Thanks.
Crazy editorial board? And the New York Times is perfectly normal to you, correct?
Oh my god you rock! Thanks for this!
FT.com has exactly the same policy – 5 free articles if being directed from Google.
However, Referer Control does not work for FT.com. Any ideas why?
This was an amazing loophole at the time I discovered it, but it has recently stopped working for me. Anyone else experiencing issues with this technique?
Still works for me as of today. Did you try the tips further abover re cookies etc?
I am having the same problem, maybe they blocked the loop
To utilize the feeling lucky search make sure to add “site:www.wsj.com” to the search.
var searchString = “Name+of+article+site%3Awww.wsj.com”
window.open(‘http://google.com/search?btnI=1&q=’+searchString,’_blank’);
So that would look like:
var searchString = “Big+Banks+Struggle+to+Pass+Feds+Stress+Tests+site%3Awww.wsj.com”;
window.open(‘http://google.com/search?btnI=1&q=’+searchString,’_blank’);
You may need to add some string replacement on the article title to add escape characters or remove apostrophes and quotes. Apostrophes and quotes will cause issues unless they are handled.
I have tested this with about 20 random articles now…works every time.
Just to be clear, this does not skirt the 5 article limit (unless you are on a VPN or can change your IP in some way (like 3g/4g users or anyone with dynamic addressing)).
I made a page that lets your put the article title in it and then does the search through a google ‘I’m feeling lucky’ search. Its a simple page…nothing fancy, took about 15 minutes.
http://www.hutchinssite.com/wsj/
The block after 5 times is based on data stored in cookies, not the IP address like I thought. My testing setup is a 4g tablet that I turned off and on which not only reset my IP (from a pool), it also cleared my cookies. I have tested using a VPN and it does not help. Opening up my cookies though and removing all cookies for the wsj.com domain clears the issue and allows for more views than 5 (5 more until the cookies need to be cleared again.) I am going to see if I can update that little page I made to also look at your cookies and try to clear them, or at least the offending cookie. That way, their paywall would effectively never stop you from viewing an article. Should be done in a day or so.
Unfortunately, javascript is not going to allow that either. Same origin policy will stop me from reading their cookies…should have thought about that!
Anyway, its possible with something like an applet, but I am not going to go through the trouble of making something that by default most users would have to click and accept 3 or 4 times just to get it running (as is the case with most java applets of that nature).
For now, you can use the page I setup if you want or continue doing to use the referrer method. In either case, if you begin to see that articles are not working again (after 5 articles), go into your cookies (FireFox is Options–>Privacy–>ShowCookies, Then type wsj.com) and remove any wsj.com cookies you see. This will allow you to then view more articles.
Talk about headaches. It goes deeper. It appears to be a mix of cookies and IP. Using a proxy appears to not work and maybe they have a bank with known proxies or they are figuring the address prior to the proxy in some way. I will test this more on my home machine as I have a few different VPN’s (not simply proxies) that I can test from there. However, if it ends up being that they are watching proxies or getting the origin ip somehow, then a VPN may be the only viable solution along with clearing cookies. That makes the whole process need a setup (setting up a VPN) and that is outside of what I can do from a website.
I’ll post back with what I find likely next week when I am back at work.
I’m having issues, too.
“but I can?t bring myself to give money to Murdoch or anyone on their crazy editorial board”
So you’re a freaking leech like all those scmbags who voted for Obama?
Thanks for this. Also, by the way, it’s “lo and behold” not “low and behold”.
Has the WSJ changed how their paywall works recently? (as in early 2016) Using Google as the referrer no longer gets me access to stories behind the paywall.
Thanks for sharing this – what an awesome trick. I was skeptical at first, but it really works!!
Cheers mate
This trick no longer works :/
UPDATE!
I found a new trick that works for WSJ, Financial Times and other paywall sites!
It works using a facebook re-direct link- if you click on these articles through a facebook page they will all open with no paywall. 🙂
I’m going to write a guide about this at Fisnowball.com, or DM me on Twitter if you want the details! @fisnowball