[PyQt] headless browsing via PyQT4

[Andreas Pakulat]

Hi,

On Sun, Feb 17, 2013 at 1:27 AM, Jiangcheng Bao <jbao605 at gmail.com> wrote:
> I am trying to achieve headless browsing via QWebView or QWebPage, but
> looks like the application would require a X server to connect to,
> even if I don't call the .show() method.
>
> The sample code at
> http://webscraping.com/blog/Scraping-JavaScript-webpages-with-webkit/,
> that does not call .show() method, but would fail with the following
> error when running under terminal without DISPLAY set actually:
>
> cannot connect to X server
>
> I also tried to use module webkit.WebkitBrowser on
> http://webscraping.com/, by creating WebkitBrowser and set gui=False
> in the init method, but it also fails with the same error when there's
> no DISPLAY.
>
> Anybody has some idea how do I actually achieve headless browsing, by
> either subclass QWebView, or webkit.WebkitBrowser?

At least for QWebView the answer is you can't. You can use Xvnc or
something similar which provides an X11 server without requiring an
actual monitor.

But maybe what you really want is: http://phantomjs.org/ which is a
headless webkit.

Andreas