I was recently introduced to a really cool domain flyover tool named Aquatone written by Michael Henriksen. In the past, I had used another really cool tool known as EyeWitness to perform similar tasks. In short, both of these tools screenshot web application resources (among other things) and make it really easy to get a quick visual representation of web page.
I’ll often look to screenshot numerous URLs while working on a network penetration test or assessing large applications. Basically, whenever I have a lot of URLs to visit with the reasons for visiting them being:
- I need to put my eyeballs on a web page and quickly figure out what purpose it serves.
- I want to capture the request and response in Burp Suite.
The manner in which I typically end up with a large number of un-visited URLs is after I’ve manually enumerated an application. When faced with a larger application it’s sometimes not altogether reasonable to manually click on “all the things” and stare at them for a bit. Given a short assessment period and hundreds of pages to access, you sometimes have to accept that you’ll only have the bandwidth to work on sample of the application’s surface-area.
A technique I like to employ in efforts to reduce missing portions of an application’s surface-area has me using the “Copy Links” right-click functionality found in Burp’s Proxy History in conjunction with Aquatone. Ensuring you’ve filtered your Proxy History to only include in-scope links that you want to enumerate, select all (control-a) of the displayed history, right-click, choose “Copy Links”, and finally write out your copy buffer to a text file.
Perform a little command-line fu on the file contents until you’re satisfied with the results.
# sort -u /tmp/links.txt | grep 'www.neversummer.com' | grep -vE '\.(svg|png|jpg|jpeg|css)’ https://www.neversummer.com/ https://www.neversummer.com/apparel https://www.neversummer.com/apparel?pagenumber=2 https://www.neversummer.com/apparel?pagenumber=3 https://www.neversummer.com/big-gun https://www.neversummer.com/dipstick https://www.neversummer.com/events https://www.neversummer.com/maverix https://www.neversummer.com/rad-slash-short-sleeve https://www.neversummer.com/snowboards https://www.neversummer.com/Team/13/Mikey-Marohn https://www.neversummer.com/Team/14/Nick-Larson https://www.neversummer.com/Team/18/Chase-Blackwell https://www.neversummer.com/Team/19/Taylor-Pfaff https://www.neversummer.com/Team/20/Matt-Williams https://www.neversummer.com/Team/21/Cooper-Sclar https://www.neversummer.com/westbound
And then pipe the contents into Aquatone for some awesome screenshots!
Note that Aquatone defaults to using port 80, 443, 8000, 8080, and 8443. You can alter this behavior in a number of ways. The easiest of which is use the “-ports” switch and provide a comma-separated string of ports. Reference the docs for more info on this switch.
You can also append the port number to the URL and Aquatone will pick up on it.
Opening up the report gives you a great visual representation of the URLs you just requested.
I’m a big fan of the web component and header summations.
Where the Burp portion of this post comes in is when I want to perform a flyover in an authenticated manner. When given credentials to an application I am assessing, I put a good deal of effort into ensuring I scan all the things in an authenticated manner. I do this by building Session Handling rules that allow Burp Suite to maintain an authenticated state while performing any sort of automated attacks (Repeater, Intruder, Scanner, etc.). I make sure I’m sending CSRF tokens in my requests in the same manner, as well.
In short, I often use Burp Suite as an authentication engine for command-line tools. This is accomplished by adding the Proxy to the scope of any configured Session Handling rules (Cookie Jar, Login, CSRF Tokens, etc.).
You’ll also need to point whatever tool you’re using at one of Burp’s proxy listeners. With Aquatone that looks something like:
# cat urls.txt | aquatone -proxy http://127.0.0.1:8081
Which will successfully populate your Burp project’s Proxy History, allow you to apply Session Handling Rules, and screenshot your list of URLs with Aquatone. Sweet!
It’s a beautiful, well-explained article. I think grep should be added in these extensions=> gif,ico,woff,eot,ttf
grep -vE ‘\.(svg|png|jpg|ico|gif|jpeg|css|woff|eot|ttf)’
Thanks, Cihan!