How to estimate proxy traffic before you buy
Proxy traffic is billed by the gigabyte, so the size of a job is arithmetic. The only number you need to measure is the average response size.
Step 1. Measure real pages
Take ten to twenty pages typical of the job. If you use an HTTP client, this cURL command prints the bytes downloaded for one URL:
curl -s -o /dev/null -w "%{size_download}\n" "$URL"
If you use a browser, open the Network tab in developer tools, reload the page and read the transferred total at the bottom. Take the average, not the smallest page.
Step 2. Multiply
GB = average size × requests per run × runs per day × days
Say a job fetches 5,000 pages of 200 KB, twice a day, for 30 days. That is 200 KB × 5,000 × 2 × 30 = 60,000,000 KB, or 60 GB with decimal units. These figures are an example, not typical values. The bandwidth calculator does the same sum.
Step 3. Add a margin
The estimate covers successful responses. Retries, redirects and request headers add to it, by an amount that depends on your target. Run the job on a small sample and compare the dashboard's usage with the estimate. That gives you a correction factor that beats any rule of thumb.
How to use less
- Prefer an HTTP client to a browser when the data is in the HTML. A browser also downloads scripts, fonts and images.
- Block heavy resources if you do need a browser. In Playwright, abort requests for images, media and fonts.
- Accept compressed responses. Most clients do this, but check that yours sends
Accept-Encoding. - Skip what has not changed. Conditional requests with
If-None-Matchreturn no body when a page is unchanged, if the site supports them. - Fetch less often. Check what the decision really needs: hourly checks use 24 times the traffic of daily ones.
Buying for the estimate
Unused traffic has no expiry date, and a top-up adds to the same plan, so there is no need to buy for the worst case. Start small, measure, then buy what the measurement supports. Current prices.