Sunday, October 19, 2008

Helpful Scripts For Work

I can be pretty lazy at times ... but in a good way, I think. I am a firm believer in the philosophy heard numerous times in various forms: "If you have to do something more than twice, have a computer do it". While I don't strive to find tons of little shortcuts to daily life, I do enjoy them.

At work in the past week, two scripts have saved me countless seconds maybe even minutes, and I figured I would share them here:

DZone RSS Links Streamliner

I use Google Reader to peruse the zillions of RSS feeds I am subscribed to and find the many articles I want to read and the fewer that I actually find time to. is a really nice article aggregator centered on tech and programming news from around the web (think Digg before it got way overcrowded and full of celebrity gossip posts and links to fark ... not that there isn't a need for them).

Often I will find several DZone links I want to read at once and will open them all into new tabs from Google Reader and then read them one by one ... only one small problem ... when the link opens its to the DZone page. This is fine except that I currently don't have a DZone account so I don't vote on storries and I rarely read the comments ... I just want the article (e.g. here). Obviously having to click twice is at least 1 too much ;-)

Thus, here is a GreaseMonkey script I added so that I don't have to click through that extra time:

// ==UserScript==
// @name DZone Forward
// @namespace yardspoon
// @description Forwards user past DZone rss link page
// @include*
// ==/UserScript==

window.location = document.evaluate("//div[@class='ldTitle']/a/@href", document, null, XPathResult.STRING_TYPE, null).stringValue;
It uses FF's built in XPath evaluator to find a link to the real article and forward the browser on to the page. It might be overkill for 1 click, but I like it. I don't have anything agains't DZone, I just don't like clicking more than I need to.

User-Agent Check Cheater

Most of my career has been working on networked systems and applications ... most of which have had a web frontend. Despite the best laid plans, often I am left with a situation where I really need to see whats actually being sent back between the server and the browser/client/web service and that can be difficult to do depending on the information you want to see. FF plugins like FireBug, XPather, and LiveHTTPHeaders give you tons of good information but can't give you the whole picture of whats being sent over the wire.

Enter WebScarab. I found this Java based personal web proxy right out of college while at my first client and am still finding uses for it frequently. After starting up a listener in WebScarab and configuring your browser to use it as a web proxy, it can easily monitor and display information about every single request and response sent (HTTP headers, redirects, content over the wire, additional resource requests, timing, etc ... great of AJAX debugging). Along with viewing the content it has numerous other features that allow you to simulate network conditions, manually intercept requests/responses before they are sent, do security fuzzy logic tests, test web services and much more.

One of the less used features but very important this week was the ability to act on requests and responses directly in a scripting language (BeanShell). This past week I was trying to install an application that for whatever reason, during the install process, needed to download additional information from the company's website. Only one problem: the installer uses IE not the default browser to do this, and the company blocks IE for external use for security reasons (FF installed on every desktop). This meant the installer failed. I could put in a support request for a temporary hole in the network to allow IE to access this site but since it was such a small thing I figured I would try to solve the problem directly.

It turns out outgoing requests are simply filtered by their user-agent HTTP header value (effective for IE blocking and simple) but also simply circumvented.

Queue WebScarab, the following script overrites the user-agent header of all requests with the value for FF ... meaning IE works and I was able to install the software. I turned the script back off when I was done so as to not risk any security issues and not piss off the admins ... but it worked none the less.
import org.owasp.webscarab.model.Request;
import org.owasp.webscarab.model.Response;
import org.owasp.webscarab.httpclient.HTTPClient;

public Response fetchResponse(HTTPClient nextPlugin, Request request) throws IOException {
request.setHeader("User-Agent", "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv: Gecko/2008092417 Firefox/3.0.3");
response = nextPlugin.fetchResponse(request);
return response;
One other great feature of WebScarab is that if you set IE to use the web proxy, your really setting the default proxy for connections by other Windows applications that use the defaults ... meaning you can suddenly see all of the traffic that random applications are sending without you knowing ... apps phoning home, checking for updates, reporting usage statistics, etc. Kinda scarry.

Also ... watch-out ... the request and responses can be seen in plain text and/or Base64 Encoded text can be converted .... so don't send your password through it unless your paying attention :-) and realize how much of your web interactions are sent unencrypted over the wire for anyone who wants to listen to find.

No comments: