# Unified Web Search API One simple search endpoint, fuelled by the group's pooled Bright Data free-tier accounts with a sponsor-funded serper.dev account as backup. Callers never know or care which account served them. Base URL (local): `http://127.0.0.1:8787` --- ## Authentication A single API key, kept in `config/api-key.json`: ```json { "key": "serpapi_change_me_0000000000000000" } ``` Change that file whenever you like — the server re-reads it automatically, no restart needed. Enforcement is controlled by `auth.enabled` in `config/settings.json` (or the `AUTH_ENABLED` env var), so it stays off while you develop locally and on once it is hosted. Send it as a header (preferred) or a query parameter: ``` X-API-Key: your_key_here ?api_key=your_key_here ``` `/health` is never authenticated — it is what a load balancer probes. --- ## `GET|POST /v1/search` | Parameter | Type | Default | Notes | | --- | --- | --- | --- | | `q` | string | **required** | The query. Max 400 characters. | | `gl` | 2-letter country | `us` | Country to search from. **Always send this** — see *Geography* below. | | `hl` | language | `en` | Interface language. | | `page` | 1–10 | `1` | ~10 results per page. | ```bash curl "http://127.0.0.1:8787/v1/search?q=best+pizza+nyc&gl=us&hl=en" curl -X POST http://127.0.0.1:8787/v1/search \ -H 'content-type: application/json' \ -d '{"q":"python asyncio event loop","gl":"us","hl":"en","page":1}' ``` ### Success ```json { "ok": true, "result": { "query": "python asyncio event loop", "page": 1, "gl": "us", "hl": "en", "results": [ { "title": "A Conceptual Overview of asyncio", "url": "https://docs.python.org/3/howto/a-conceptual-overview-of-asyncio.html", "snippet": "The event loop takes a job from its backlog of work and invokes it...", "source": "Python documentation", "position": 1 } ], "peopleAlsoAsk": [{ "question": "...", "snippet": "...", "url": "..." }], "relatedSearches": ["python asyncio event loop example"], "totalResults": 123, "resolvedCountry": "US", "servedBy": { "provider": "brightdata", "account": "Mourad Oueslati", "role": "free" }, "tookMs": 4820 } } ``` `position` is **absolute across pages** — page 2 starts at 11, not at 1. (serper.dev restarts its own counter on every page; that is normalised away for you.) ### Refusals A refusal is **data in a `200` response**, never an HTTP error code. A "no capacity right now" answer carries a machine-readable time to come back, and mapping it onto a 429 or 503 would throw that away and invite every proxy between us to retry on its own terms. ```json { "ok": false, "error": { "code": "CAPACITY", "message": "no source has capacity", "retryAtMs": 1788372000000, "unavailable": [{ "sourceId": "bd:acc_...", "reasons": ["limit:0"], "retryAt": 1788372000000 }] } } ``` A `Retry-After` header is set alongside it for clients that prefer headers. | `error.code` | Meaning | What to do | | --- | --- | --- | | `CAPACITY` | Every account is out of credit or rate-limited right now. | Retry at `retryAtMs`. | | `QUEUE_TIMEOUT` | Waited its allowance and gave up. | Retry at `retryAtMs`. | | `ALL_ATTEMPTS_FAILED` | Every account tried and failed. `attempts[]` says what each said. | Inspect `attempts`; a 4xx there means the query itself is bad. | | `NO_ELIGIBLE_SOURCE` | The pool is empty or every account is paused. | Check `/v1/stats`; this is an operator problem. | | `BAD_REQUEST` / `BAD_JSON` | Bad parameters. Returned as HTTP 400. | Fix the request. | | `UNAUTHORISED` | Missing or wrong key. Returned as HTTP 401. | Check `X-API-Key`. | --- ## `GET /v1/stats` Live pool health: available concurrency and roughly how many searches the pool can still serve before the calendar month ends. ```bash curl http://127.0.0.1:8787/v1/stats ``` ```json { "ok": true, "calendarMonth": { "end": "2026-10-01T00:00:00.000Z", "daysRemaining": 28.3 }, "routing": { "mode": "free-only", "freeAccountsWithQuota": 3, "blendThresholdAccounts": 2, "backupShareInBlend": 0 }, "concurrency": { "availableNow": 17, "total": 17, "inFlight": 0, "queued": 0 }, "credits": { "free": { "remaining": 14993, "capacity": 15000, "used": 7, "availableUntilMonthEnd": 14993, "accountsWithQuota": 3 }, "backup": { "provider": "serper", "balance": 2472, "autoTopUp": true }, "totalAvailableUntilMonthEnd": 17465 }, "projection": { "burnPerHour": 3.2, "hoursOfFreeCreditLeft": 4685.3, "freeExhaustionAt": null, "freeCreditsLastTheMonth": true }, "sources": [ /* per-account detail: credits, reset instant, concurrency, counters */ ], "lastAccountRefresh": { "at": "...", "ok": true, "added": [], "problems": [] } } ``` ### How to read the numbers - **`concurrency.availableNow`** — how many more searches could start *this instant* without queueing. It is the sum of each usable account's free slots. - **`credits.free.availableUntilMonthEnd`** — what the free pool can still serve before the month rolls. Free credit does not roll over, so this is normally just what is left; if an account's own cycle happens to reset inside the horizon, that refill is counted too. - **`credits.totalAvailableUntilMonthEnd`** — the free estimate plus the backup's current balance. The backup has auto top-up, so treat it as a floor, not a cap. - **`projection`** — extrapolates the last 24 hours of real traffic. `freeCreditsLastTheMonth: false` is the signal to add accounts or raise the sponsor's budget. - **`sources[].credits.resetsAt`** — each account's own reset instant. **These differ per account** (see below). --- ## How routing works All accounts sit in one pool. For each search: | Free accounts still holding credit | What happens | | --- | --- | | 3 or more | **Free only.** The backup stays completely cold. | | 2 | **Blend** — 50 / 50 between the backup and the free pool. | | 1 | **Blend** — 50 / 50. | | 0 | **Backup only**, until an account's cycle resets. | Within the free pool, accounts are drained **one at a time** (lowest profile number first) rather than evenly. That is deliberate: it makes "2 accounts still have credit" a gradual, meaningful signal. If load were spread evenly, every account would empty at nearly the same moment and the pool would jump straight from "all healthy" to "all empty", never passing through the blend zone. The consequence worth knowing: within a month, the first member's account carries most of the traffic and the last member's may go untouched. Set `pool.freeStrategy` and the per-account `order` in `config/settings.json` if you would rather rotate who goes first each month. Failover is automatic. If an account errors, the request is retried on another one before the caller ever sees a failure, up to `request.maxAttempts`. --- ## Geography — the one real caveat Bright Data uses **shared exit nodes**, so an identical query can resolve in a different country on each call unless pinned. The service always sends your `gl` as the exit country, but *city-level* location still floats on both providers. - For informational queries the two providers were measured returning **identical results in identical order**. - For local-intent queries ("pizza near me", "plumber"), expect different cities between calls regardless of provider. Always send `gl`, and do not rely on city-level consistency. `result.resolvedCountry` tells you where the search actually resolved. --- ## Limits - ~10 organic results per page. Neither provider honours a `num` parameter — Bright Data strips it outright, and serper.dev echoes it without effect. Use `page`. - `page` maxes out at 10. - Per-account rate limits are enforced locally *before* a request is sent, so you get a `CAPACITY` answer with a retry time rather than a provider 429. --- ## Operations ```bash npm install node test/probe-accounts.mjs # verify every account + proxy. Spends 0 credits. npm start # start the API ``` - **New team members** are picked up automatically: the group vault is re-read every hour, and new accounts are probed (0 credits) before joining the pool. Accounts that disappear from the vault are paused, never deleted — their usage ledger is kept so they resume cleanly. - **Usage is reconciled** against each account's own `reqs_serp` counter every 10 minutes, so credit spent outside this service still counts. - **Run exactly one process** per `storageDir`. Two would keep two independent ledgers and each would believe it owned the whole group quota. A second process exits immediately with a clear message. - **Shut down with SIGINT/SIGTERM**, not SIGKILL, so the ledger is compacted and the storage lock released. A hard kill is survivable — in-flight credit is counted as spent, which is the safe direction — but a clean stop is free. ### Accounts do not all reset on the 1st Each Bright Data account's free tier resets on **its own signup anniversary**, at a wall-clock time, not at midnight on the 1st. The three current accounts reset at `Sep 1 03:44 UTC`, `Sep 2 16:14 UTC` and `Sep 2 16:24 UTC` respectively. The service reads each account's real cycle start from Bright Data and anchors that account's quota window to it, re-anchoring each cycle. This is why `/v1/stats` reports a different `resetsAt` per account, and why "credits available until the end of the calendar month" is not simply "5000 × number of accounts".