Scraping Pastebin

by {"name"=>"Stephen Haywood", "url"=>"https://twitter.com/averagesecguy"}

At Thanksgiving, Pastebin ran a special on their Lifetime Pro membership. I decided to get a membership so I could start scraping. Of course, I forgot to get the membership while the special was running so I ended up paying full price. It’s still worth it.

This week I finally did something with the membership. I built a scraper with Go and started collecting interesting pastes. The scrape tool searches for credential dumps, I’m still tweaking the regex for that, and searches for keywords. You can easily add additional keywords or new custom processing methods to the code.

The code is available on Github. As always, if you have any questions, email me or @ me on Twitter. Yes, I’m back on Twitter, @averagesecguy.