FindBroken Beta
Here’s something I’ve been working on for the last month or so: FindBroken.com. It’s an automated link checker that periodically scans your site for broken links and alerts you by email if any are found. Basically, it’s like a hosted version of the Broken Link Checker plugin, only simpler and not WordPress-specific.
The site is still in early beta, so some features may a little rough around the edges or just plain missing. Nevertheless, I encourage you to go check it out and let me know what you think. All feedback is welcome.
What already works
- Add/remove sites.
- Monitor multiple sites with one account.
- Automatic daily scans.
- Email notifications.
What doesn’t quite work
- The crawler that I use is slow to get going. Sometimes it can take ~30 minutes before any progress is visible.
- Redirects are currently not followed.
- There’s a fair amount of false positives. Luckily, most of them are temporary and don’t show up when using the “persistent” filter.
- The site could use some UX love.
Screenshots
Subscribe via RSS
Get Updates by Email
I’d like to try this but is there any chance you might offer Paypal as a payment method? I don’t like using my credit card online.
Sorry, no – the payment processor that I use for subscription processing doesn’t support PayPal.
Even though I read your comment earlier about the bot obeying robots.txt, I forgot that I block bots from following my affiliate links. I just added the following to the bottom of my robots.txt:
#FindBroken
User-agent: 80legs
Allow: /*
I’m thinking this will override the other instructions.
Good idea, but you need to use “User-Agent: FindBroken” for this bot. I moved away from 80legs some time ago as they didn’t meet my needs.
Is there anyway to exclude the FindBroken bot from Google Analytics? It’s skewing my averages…
Correct me if I’m wrong, but doesn’t GA only track user agents that can run JavaScript? Since my bot doesn’t (can’t) run JS, I had assumed it wouldn’t affect anyone’s GA stats.
Nevermind, you’re right. It was actually me that was being tracked after following links back to my site from FindBroken.com. DO’H!!
I thought I, as an admin, was excluded from my GA stats! Anyway, thanks!
I’d like to try this but is there any chance you might offer Paypal as a payment method? I don’t like using my credit card online.
I’ve considered it, but the numerous horror-stories about PayPal freezing accounts randomly always put me off.
Recently, PayPal started offering automatic settlement withdrawals–from PayPal.com “Settlement Withdrawal enables you to automatically transfer money from your PayPal account to your bank account at the end of each day.” It’s free for U.S. accounts. So, if you use this feature, you significantly reduce the risk of PayPal freezing big chunks of your money.
I just installed and enabled Bad Behavior on my site, will FindBroken continue to work without problems? Thanks!
Sorry, I haven’t tested it with BB. Lets wait and see how it goes, I guess.
It’s been awhile since I last received any reports from FindBroken. Could that be because of BB?
Everything looks good from what I can see. In any case, I would expect BB to lead to some working links being reported as broken, not the other way around.
My understanding is that BB blocks requests that might not be legitimate. It examines headers and whatnot, so I was worried that the FindBroken wasn’t looking legitimate enough to BB and thus was blocked.
Yep, but if it does get blocked, it should start showing all of those (blocked) links as broken, which would make it send more reports, not less.
Hey, Would be nice if FindBroken had a way to ignore/hide broken URLs in comment author names—those are difficult to keep clean. I don’t think broken links in them impact user experience or a web page ranking negatively…
If your comment author links are tagged with rel=”nofollow” (like e.g. WordPress does), you can make FB skip them by enabling the “Ignore nofollow links” option when adding a site.
This sucks. Why? What’s a good alternative?
Plans didn’t pan out, financial-wise. Also personal reasons (which I’d rather not go into details of).
As for alternatives, there’s the very popular Xenu’s Link Sleuth (a free Windows app) and a bunch of commercial tools like http://www.linktiger.com/ and http://www.linkalarm.com/