Monitor web sites visited using Internet Explorer, Opera, Chrome, Firefox and Safari in Python

Question

I am working on a project for work and have seemed to run into a small problem. The project is a similar program to Web Nanny, but branded to my client's company. It will have features such as website blocking by URL, keyword and web activity logs. I would also need it to be able to "pause" downloads until an acceptable username and password is entered.

I found a script to monitor the URL visited in Internet Explorer (shown below), but it seems to slow the browser down considerably. I have not found any support or ideas onhow to implement this in other browsers.

So, my questions are:

1). How to I monitor other browser activity / visited URLs? 2). How do I prevent downloading unless an acceptable username and password is entered?

---

from  win32com.client import Dispatch,WithEvents
import time,threading,pythoncom,sys

stopEvent=threading.Event()
class EventSink(object):

    def OnNavigateComplete2(self,*args):
        print "complete",args
        stopEvent.set()


def waitUntilReady(ie):
    if ie.ReadyState!=4:
        while 1:
            print "waiting"
            pythoncom.PumpWaitingMessages()
            stopEvent.wait(.2)
            if stopEvent.isSet() or ie.ReadyState==4:
                stopEvent.clear()
                break;

time.clock()
ie=Dispatch('InternetExplorer.Application',EventSink)
ev=WithEvents(ie,EventSink)
ie.Visible=1
ie.Navigate("http://www.google.com")

waitUntilReady(ie)
print "location",ie.LocationName
ie.Navigate("http://www.aol.com")
waitUntilReady(ie)
print "location",ie.LocationName
print ie.LocationName,time.clock()
print ie.ReadyState

Answer 1

I would recommend looking into a nice web proxy. If the machines are all on the same network you can implement a transparent caching web proxy and put filtering rules on it. They tend to be high speed and can do lots of cool things.

I have had some luck with Squid. Would this solve your situation?

Answer 2

You need to implement this as a C++ BHO, sink DWebBrowserEvents2::OnBeforeNavigate and implement your logic there as it is a place that will block the navigate synchronously until you return, and you can cancel the navigation there as well.