Revision: 59324
Initial Code
Initial URL
Initial Description
Initial Title
Initial Tags
Initial Language
at September 1, 2012 07:15 by scrapy
Initial Code
# This recipe was taken from the old wiki. # # You can make Scrapy send HTTP/1.1 requests by overriding the Scrapy HTTP Client Factory, with the following (undocumented) setting: # # DOWNLOADER_HTTPCLIENTFACTORY = 'myproject.downloader.HTTPClientFactory' from scrapy.core.downloader.webclient import ScrapyHTTPClientFactory, ScrapyHTTPPageGetter class PageGetter(ScrapyHTTPPageGetter): def sendCommand(self, command, path): self.transport.write('%s %s HTTP/1.1 ' % (command, path)) class HTTPClientFactory(ScrapyHTTPClientFactory): protocol = PageGetter # Snippet imported from snippets.scrapy.org (which no longer works) # author: pablo # date : Sep 16, 2011
Initial URL
Initial Description
Initial Title
Spoof requests as HTTP/1.1
Initial Tags
Initial Language
Python