|
| 1 | +--- |
| 2 | +output: hugodown::hugo_document |
| 3 | + |
| 4 | +slug: httr2-1-1-0 |
| 5 | +title: httr2 1.1.0 |
| 6 | +date: 2025-01-20 |
| 7 | +author: Hadley Wickham |
| 8 | +description: > |
| 9 | + httr2 1.1.0 introduces powerful new streaming capabilities with |
| 10 | + `req_perform_connection()`, as well as comprehensive URL |
| 11 | + manipulation tools, improved AWS support, and a bunch of bug fixes. |
| 12 | +
|
| 13 | +photo: |
| 14 | + url: https://unsplash.com/photos/person-holding-two-baseballs-3k_FcShH0jY |
| 15 | + author: Jose Morales |
| 16 | + |
| 17 | +# one of: "deep-dive", "learn", "package", "programming", "roundup", or "other" |
| 18 | +categories: [package] |
| 19 | +tags: [httr2] |
| 20 | +--- |
| 21 | + |
| 22 | +<!-- |
| 23 | +TODO: |
| 24 | +* [x] Look over / edit the post's title in the yaml |
| 25 | +* [x] Edit (or delete) the description; note this appears in the Twitter card |
| 26 | +* [x] Pick category and tags (see existing with `hugodown::tidy_show_meta()`) |
| 27 | +* [x] Find photo & update yaml metadata |
| 28 | +* [x] Create `thumbnail-sq.jpg`; height and width should be equal |
| 29 | +* [x] Create `thumbnail-wd.jpg`; width should be >5x height |
| 30 | +* [x] `hugodown::use_tidy_thumbnails()` |
| 31 | +* [x] Add intro sentence, e.g. the standard tagline for the package |
| 32 | +* [x] `usethis::use_tidy_thanks()` |
| 33 | +--> |
| 34 | + |
| 35 | +We're chuffed to announce the release of [httr2 1.1.0](https://httr2.r-lib.org). httr2 (pronounced "hitter2") is a comprehensive HTTP client that provides a modern, pipeable API for working with web APIs. It builds on top of [{curl}](https://jeroen.r-universe.dev/curl) to provide features like explicit request objects, built-in rate limiting & retry tooling, comprehensive OAuth support, and secure handling of secrets and credentials. |
| 36 | + |
| 37 | +In this blog post, we'll dive into the new streaming interface built around `req_perform_connection()`, explore the new suite of URL manipulation tools, and highlight a few of the other biggest changes (including better support for AWS and enhancements to the caching system), and update you on the lifecycle changes. |
| 38 | + |
| 39 | +This blog post includes the most important enhacenments in versions 1.0.1 through 1.0.7, where we've been iterating on various features and fixing _numerous_ bugs. For a complete list of changes, you can check the [GitHub release notes](https://github.com/r-lib/httr2/releases) or the [NEWS file](https://httr2.r-lib.org/news/index.html). |
| 40 | + |
| 41 | +## Installation |
| 42 | + |
| 43 | +Install httr2 from CRAN with: |
| 44 | + |
| 45 | +```{r} |
| 46 | +#| eval: false |
| 47 | +install.packages("httr2") |
| 48 | +``` |
| 49 | + |
| 50 | +## Streaming data |
| 51 | + |
| 52 | +The headline feature of this release is a better API for streaming responses, where the body is not available immediately but is streamed back over time. This is particularly important for interacting with LLMs, where it's needed to make chat responses feel snappy. You can try it out in [ellmer](https://ellmer.tidyverse.org), our new package for chatting with LLMs from a variety of providers. |
| 53 | + |
| 54 | +The most important new function is `req_perform_connection()`, which supersedes the older callback-based `req_perform_stream()`. Unlike its predecessor, `req_perform_connection()` returns a regular response object with a connection object for the body: |
| 55 | + |
| 56 | +```{r} |
| 57 | +library(httr2) |
| 58 | +
|
| 59 | +req <- request(example_url()) |> req_template("/stream-bytes/:n", n = 10240) |
| 60 | +resp <- req_perform_connection(req) |
| 61 | +resp |
| 62 | +``` |
| 63 | + |
| 64 | +Once you have a streaming connection you can repeatedly call a `resp_stream_*()` function to pull down data in chunks, using `resp_stream_is_complete()` to figure out when to stop. |
| 65 | + |
| 66 | +```{r} |
| 67 | +while (!resp_stream_is_complete(resp)) { |
| 68 | + bytes <- resp_stream_raw(resp, kb = 2) |
| 69 | + cat("Downloaded ", length(bytes), " bytes\n", sep = "") |
| 70 | +} |
| 71 | +``` |
| 72 | + |
| 73 | +As well as `resp_stream_raw()`, which returns a raw vector, you can use `resp_stream_lines()` to stream lines and `resp_stream_sse()` to stream [server-sent events](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events). |
| 74 | + |
| 75 | +## URL manipulation tools |
| 76 | + |
| 77 | +Working with URLs got easier with three new functions: `url_modify()`, `url_modify_query()`, and `url_modify_relative()`. You can see how they work in the examples below: |
| 78 | + |
| 79 | +```{r} |
| 80 | +# url_modify() modifies components of a URL |
| 81 | +url_modify("https://example.com", hostname = "github.com") |
| 82 | +url_modify("https://example.com", scheme = "http") |
| 83 | +url_modify("https://example.com", path = "abc", query = list(foo = "bar")) |
| 84 | +
|
| 85 | +# url_modify_query() lets you modify individual query parameters |
| 86 | +# modifying an existing parameter: |
| 87 | +url_modify_query("http://example.com?a=1&b=2", a = 10) |
| 88 | +# delete a parameter: |
| 89 | +url_modify_query("http://example.com?a=1&b=2", b = NULL) |
| 90 | +# add a new parameter: |
| 91 | +url_modify_query("http://example.com?a=1&b=2", c = 3) |
| 92 | +
|
| 93 | +# url_modify_relative() navigates to a relative URL |
| 94 | +url_modify_relative("https://example.com/a/b/c.html", "/d/e/f.html") |
| 95 | +url_modify_relative("https://example.com/a/b/c.html", "C.html") |
| 96 | +url_modify_relative("https://example.com/a/b/c.html", "../B.html") |
| 97 | +``` |
| 98 | + |
| 99 | +We also added `req_url_relative()` to make it easier to navigate to a relative URL for an existing request. |
| 100 | + |
| 101 | +## Other improvements |
| 102 | + |
| 103 | +There are a handful of other improvements that are worth highlighting: |
| 104 | + |
| 105 | +* We've made it easier to talk to AWS web services with `req_auth_aws_v4()` for signing requests and `resp_stream_aws()` for streaming responses. Special thanks goes to the [lifion-aws-event-stream](https://github.com/lifion/lifion-aws-event-stream/) project for providing a clear reference implementation. |
| 106 | + |
| 107 | +* We've run-down a long list of bugs that made `req_cache()` unreliable. This includes improving the handling of header-only changes, better cache pruning, and new debugging options. If you're working with a web API that supports caching, we highly recommend that you try it out. The next release of {[gh](https://github.com/r-lib/gh)} will use a cache by default, and my use of the dev version suggests that it gives a pretty nice performance improvment. |
| 108 | + |
| 109 | +* `is_online()` provides an easy way to check internet connectivity. |
| 110 | + |
| 111 | +* `req_perform_promise()` allows you to execute requests in the background (thanks to [@gergness](https://github.com/gergness)) using an efficient approach that waits on curl socket activity (thanks to [@shikokuchuo](https://github.com/shikokuchuo)). |
| 112 | + |
| 113 | +## Breaking changes |
| 114 | + |
| 115 | +As httr2 continues to mature, we've made some lifecycle changes: |
| 116 | + |
| 117 | +* `req_perform_iterative()` is now stable and no longer experimental. |
| 118 | +* `req_perform_stream()` is superseded by `req_perform_connection()`, as mentioned above. |
| 119 | +* `with_mock()` and `local_mock()` are defunct and will be rmeoved in the next release. Use `with_mocked_responses()` and `local_mocked_responses()` instead. |
| 120 | + |
| 121 | +## Acknowledgements |
| 122 | + |
| 123 | +A big thanks to all 76 folks who filed issues, created PRs and generally helped to make httr2 better! [@Aariq](https://github.com/Aariq), [@AGeographer](https://github.com/AGeographer), [@amael-ls](https://github.com/amael-ls), [@anishjoni](https://github.com/anishjoni), [@asadow](https://github.com/asadow), [@atheriel](https://github.com/atheriel), [@awpsoras](https://github.com/awpsoras), [@billsanto](https://github.com/billsanto), [@bonushenricus](https://github.com/bonushenricus), [@botan](https://github.com/botan), [@burgerga](https://github.com/burgerga), [@CareCT](https://github.com/CareCT), [@cderv](https://github.com/cderv), [@cole-brokamp](https://github.com/cole-brokamp), [@covid19ec](https://github.com/covid19ec), [@datapumpernickel](https://github.com/datapumpernickel), [@denskh](https://github.com/denskh), [@deschen1](https://github.com/deschen1), [@DyfanJones](https://github.com/DyfanJones), [@erydit](https://github.com/erydit), [@exetico](https://github.com/exetico), [@fh-mthomson](https://github.com/fh-mthomson), [@frzambra](https://github.com/frzambra), [@gergness](https://github.com/gergness), [@GreenGrassBlueOcean](https://github.com/GreenGrassBlueOcean), [@guslipkin](https://github.com/guslipkin), [@hadley](https://github.com/hadley), [@i2z1](https://github.com/i2z1), [@isachng93](https://github.com/isachng93), [@IshuaWang](https://github.com/IshuaWang), [@JamesHWade](https://github.com/JamesHWade), [@jameslairdsmith](https://github.com/jameslairdsmith), [@JBGruber](https://github.com/JBGruber), [@jcheng5](https://github.com/jcheng5), [@jeroen](https://github.com/jeroen), [@jimbrig](https://github.com/jimbrig), [@jjesusfilho](https://github.com/jjesusfilho), [@jl5000](https://github.com/jl5000), [@jmuhlenkamp](https://github.com/jmuhlenkamp), [@jonthegeek](https://github.com/jonthegeek), [@JosiahParry](https://github.com/JosiahParry), [@jwimberl](https://github.com/jwimberl), [@krjaworski](https://github.com/krjaworski), [@m-muecke](https://github.com/m-muecke), [@maarten-vermeyen](https://github.com/maarten-vermeyen), [@MarekGierlinski](https://github.com/MarekGierlinski), [@maxsutton](https://github.com/maxsutton), [@mgirlich](https://github.com/mgirlich), [@MichaelChirico](https://github.com/MichaelChirico), [@mkoohafkan](https://github.com/mkoohafkan), [@MSHelm](https://github.com/MSHelm), [@mstei4176](https://github.com/mstei4176), [@mthomas-ketchbrook](https://github.com/mthomas-ketchbrook), [@NateNohling](https://github.com/NateNohling), [@nick-youngblut](https://github.com/nick-youngblut), [@pbulsink](https://github.com/pbulsink), [@PietrH](https://github.com/PietrH), [@pkautio](https://github.com/pkautio), [@plietar](https://github.com/plietar), [@pmlefeuvre-met](https://github.com/pmlefeuvre-met), [@rkrug](https://github.com/rkrug), [@romainfrancois](https://github.com/romainfrancois), [@salim-b](https://github.com/salim-b), [@shikokuchuo](https://github.com/shikokuchuo), [@simplyalexander](https://github.com/simplyalexander), [@sluga](https://github.com/sluga), [@stefanedwards](https://github.com/stefanedwards), [@steveputman](https://github.com/steveputman), [@tebancr](https://github.com/tebancr), [@thohan88](https://github.com/thohan88), [@tony2015116](https://github.com/tony2015116), [@toobiwankenobi](https://github.com/toobiwankenobi), [@verhovsky](https://github.com/verhovsky), [@walinchus](https://github.com/walinchus), [@werkstattcodes](https://github.com/werkstattcodes), and [@zacdav-db](https://github.com/zacdav-db). |
0 commit comments