Skip to content

Commit 3376a13

Browse files
committed
Update proxy section on readme
1 parent e32c250 commit 3376a13

File tree

1 file changed

+54
-39
lines changed

1 file changed

+54
-39
lines changed

README.md

Lines changed: 54 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -248,45 +248,6 @@ class AwesomeSpiderWithPage(scrapy.Spider):
248248
Scrapy request workflow (Scheduler, Middlewares, etc).
249249

250250

251-
## Proxy support
252-
253-
Proxies are supported at the Browser level by specifying the `proxy` key in
254-
the `PLAYWRIGHT_LAUNCH_OPTIONS` setting:
255-
256-
```python
257-
PLAYWRIGHT_LAUNCH_OPTIONS = {
258-
"proxy": {
259-
"server": "http://myproxy.com:3128",
260-
"username": "user",
261-
"password": "pass",
262-
},
263-
}
264-
```
265-
266-
You can also set proxies per context with the `PLAYWRIGHT_CONTEXTS` setting:
267-
268-
```python
269-
PLAYWRIGHT_CONTEXTS = {
270-
"default": {
271-
"proxy": {
272-
"server": "http://default-proxy.com:3128",
273-
"username": "user1",
274-
"password": "pass1",
275-
},
276-
},
277-
"alternative": {
278-
"proxy": {
279-
"server": "http://alternative-proxy.com:3128",
280-
"username": "user2",
281-
"password": "pass2",
282-
},
283-
},
284-
}
285-
```
286-
287-
See also the [upstream Playwright section](https://playwright.dev/python/docs/network#http-proxy)
288-
on HTTP Proxies.
289-
290251
## Multiple browser contexts
291252

292253
Multiple [browser contexts](https://playwright.dev/python/docs/browser-contexts)
@@ -355,6 +316,60 @@ async def parse_in_new_context(self, response):
355316
```
356317

357318

319+
## Proxy support
320+
321+
Proxies are supported at the Browser level by specifying the `proxy` key in
322+
the `PLAYWRIGHT_LAUNCH_OPTIONS` setting:
323+
324+
```python
325+
from scrapy import Spider, Request
326+
327+
class ProxySpider(Spider):
328+
name = "proxy"
329+
custom_settings = {
330+
"PLAYWRIGHT_LAUNCH_OPTIONS": {
331+
"proxy": {
332+
"server": "http://myproxy.com:3128"
333+
"username": "user",
334+
"password": "pass",
335+
},
336+
}
337+
}
338+
339+
def start_requests(self):
340+
yield Request("http://httpbin.org/get", meta={"playwright": True})
341+
342+
def parse(self, response):
343+
print(response.text)
344+
```
345+
346+
You can also set proxies per context with the `PLAYWRIGHT_CONTEXTS` setting:
347+
348+
```python
349+
PLAYWRIGHT_CONTEXTS = {
350+
"default": {
351+
"proxy": {
352+
"server": "http://default-proxy.com:3128",
353+
"username": "user1",
354+
"password": "pass1",
355+
},
356+
},
357+
"alternative": {
358+
"proxy": {
359+
"server": "http://alternative-proxy.com:3128",
360+
"username": "user2",
361+
"password": "pass2",
362+
},
363+
},
364+
}
365+
```
366+
367+
Or passing a `proxy` key when [creating a context during a crawl](#creating-a-context-during-a-crawl).
368+
369+
See also the [upstream Playwright section](https://playwright.dev/python/docs/network#http-proxy)
370+
on HTTP Proxies.
371+
372+
358373
## Executing actions on pages
359374

360375
A sorted iterable (`list`, `tuple` or `dict`, for instance) could be passed

0 commit comments

Comments
 (0)