{"openapi":"3.1.0","info":{"title":"Plumbline API","version":"1.0.0","summary":"Verified human judgement for AI teams. Plumbline, powered by QuestionPro.","description":"Plumbline, powered by QuestionPro.\n\nThe v1 API follows Prolific's v1 API (AI Task Builder datasets and batches, studies and their\ntransitions): same paths, verbs, `Authorization: Token <token>` header, status values and error body.\nA Prolific integration changes its base URL and token and keeps working. Trailing slashes are accepted\non every path (`/studies/` and `/studies` are the same).\n\nPlumbline's extras ride in `plumbline_task`, `plumbline_review`, `plumbline_progress` and\n`plumbline_quote` blocks that Prolific clients can ignore, plus three Plumbline-only endpoints:\n`/studies/{id}/timeline`, `/data-collection/batches/{id}/results` and the upload target `PUT /uploads/{token}`.\n\nRequest bodies are read leniently: unknown Prolific fields are stored and echoed back, never rejected.\nLists use Prolific's `results`, `meta.count` and `_links`, with `page` (from 1) and `per_page`\n(default 100, max 500).\n\n**Keys and scopes.** A key starts `plb_live_` or `plb_test_` (the mode of its workspace), then 40 random\ncharacters, `_` and a 6-character checksum. Keys are made and revoked in the console; `GET /plumbline/key`\ntells a script what its own key may do. Each key holds scopes: `datasets:read`, `datasets:write`,\n`batches:read`, `batches:write`, `results:read`, `studies:read`, `studies:write` and `studies:publish`\n(`POST /studies/{id}/transition`). A key made with the Full access preset holds every scope, so a\nProlific script works unchanged. Each operation below says which scope it needs; a key without it gets\n403 with a `detail` naming the scope, for example \"This key can't publish studies. It needs studies:publish.\"\n\n**Rate limits.** 600 requests a minute per live key (60 per sandbox key) and 3,000 a minute across all of\na live workspace's keys (300 for a sandbox). Every response carries `RateLimit-Limit`, `RateLimit-Remaining`\nand `RateLimit-Reset` (seconds) for whichever budget is tighter; a 429 adds `Retry-After`.\n\nEvery error, including 401, 403, 404, 429 and 501, uses the `Error` body. A 401 for a key that has expired\nor been revoked starts its `detail` with `key_expired` or `key_revoked`.\n\nThis file is the contract between work packages. It changes only with a `contract-change` issue\n(docs/BUILD.md rule 7).\n"},"servers":[{"url":"https://plumbline.mikeporter.ai/api/v1","description":"This server"},{"url":"https://api.plumbline.example/api/v1","description":"Production (domain to be decided)"},{"url":"http://localhost:3000/api/v1","description":"Local development"}],"security":[{"TokenAuth":[]}],"tags":[{"name":"Datasets"},{"name":"Task design"},{"name":"Studies"},{"name":"Results"},{"name":"Uploads"},{"name":"Keys"}],"paths":{"/data-collection/datasets":{"get":{"tags":["Datasets"],"operationId":"listDatasets","x-plumbline-scope":"datasets:read","summary":"List the workspace's datasets","parameters":[{"$ref":"#/components/parameters/WorkspaceIdQuery"},{"$ref":"#/components/parameters/Page"},{"$ref":"#/components/parameters/PerPage"}],"responses":{"200":{"description":"The datasets","content":{"application/json":{"schema":{"$ref":"#/components/schemas/DatasetList"}}}},"default":{"$ref":"#/components/responses/Error"}}},"post":{"tags":["Datasets"],"operationId":"createDataset","x-plumbline-scope":"datasets:write","summary":"Create an empty dataset in a workspace","requestBody":{"required":true,"content":{"application/json":{"schema":{"$ref":"#/components/schemas/DatasetCreate"},"example":{"name":"helpfulness-pairs-oct","workspace_id":"ws_01J9X3Q8ZK7H2D4M6N8P0R2T4V","schema":"pairwise_preference@v1"}}}},"responses":{"201":{"description":"The new dataset","content":{"application/json":{"schema":{"$ref":"#/components/schemas/Dataset"},"example":{"id":"ds_01J9X4T7QWB3V8K2M5N6P7Q8RS","name":"helpfulness-pairs-oct","workspace_id":"ws_01J9X3Q8ZK7H2D4M6N8P0R2T4V","status":"UNINITIALISED"}}}},"default":{"$ref":"#/components/responses/Error"}}}},"/data-collection/datasets/{id}":{"get":{"tags":["Datasets"],"operationId":"getDataset","x-plumbline-scope":"datasets:read","summary":"Get a dataset; poll until READY","description":"Plumbline adds row validation and detected categories.","parameters":[{"$ref":"#/components/parameters/DatasetId"}],"responses":{"200":{"description":"The dataset","content":{"application/json":{"schema":{"$ref":"#/components/schemas/Dataset"},"example":{"id":"ds_01J9X4T7QWB3V8K2M5N6P7Q8RS","name":"helpfulness-pairs-oct","status":"READY","datapoint_count":2000,"validation":{"rows_rejected":3,"errors":[{"row":418,"error":"response_b is empty"}]},"categories":{"chat":1210,"coding":402,"reasoning":388}}}}},"default":{"$ref":"#/components/responses/Error"}}}},"/data-collection/datasets/{id}/upload-url/{filename}":{"get":{"tags":["Datasets"],"operationId":"getDatasetUploadUrl","x-plumbline-scope":"datasets:write","summary":"Get a signed URL to PUT a CSV or JSONL file to","description":"The URL is valid for one hour. PUT the raw file body to it; no checksum headers are needed.\nPairwise schema columns: item_id, prompt, response_a, response_b, optional category, optional gold_answer.\nLimits in v1: 50 MB and 100,000 rows per file.\n","parameters":[{"$ref":"#/components/parameters/DatasetId"},{"name":"filename","in":"path","required":true,"schema":{"type":"string"},"example":"pairs.jsonl"}],"responses":{"200":{"description":"The signed upload URL","content":{"application/json":{"schema":{"$ref":"#/components/schemas/UploadUrl"}}}},"default":{"$ref":"#/components/responses/Error"}}}},"/data-collection/batches":{"get":{"tags":["Task design"],"operationId":"listBatches","x-plumbline-scope":"batches:read","summary":"List the workspace's batches","parameters":[{"$ref":"#/components/parameters/WorkspaceIdQuery"},{"$ref":"#/components/parameters/Page"},{"$ref":"#/components/parameters/PerPage"}],"responses":{"200":{"description":"The batches","content":{"application/json":{"schema":{"$ref":"#/components/schemas/BatchList"}}}},"default":{"$ref":"#/components/responses/Error"}}},"post":{"tags":["Task design"],"operationId":"createBatch","x-plumbline-scope":"batches:write","summary":"Create the batch that holds the task","description":"A Prolific batch body works unchanged; the `plumbline_task` block switches on the native pairwise question.","requestBody":{"required":true,"content":{"application/json":{"schema":{"$ref":"#/components/schemas/BatchCreate"},"example":{"name":"Helpfulness pairs, October","workspace_id":"ws_01J9X3Q8ZK7H2D4M6N8P0R2T4V","dataset_id":"ds_01J9X4T7QWB3V8K2M5N6P7Q8RS","task_details":{"task_name":"Which answer is better?","task_introduction":"<p>Read the prompt and both answers.</p>","task_steps":"<ol><li>Pick A, B or about the same</li></ol>"},"plumbline_task":{"type":"pairwise_preference","allow_tie":true,"rubric":["helpful","accurate","safe"],"rationale":"optional","counterbalance_order":true}}}}},"responses":{"201":{"description":"The new batch","content":{"application/json":{"schema":{"$ref":"#/components/schemas/Batch"},"example":{"id":"bt_01J9X5B2MRD7F9H1K3M5P7R9TV","status":"UNINITIALISED","dataset_id":"ds_01J9X4T7QWB3V8K2M5N6P7Q8RS"}}}},"default":{"$ref":"#/components/responses/Error"}}}},"/data-collection/batches/{id}":{"get":{"tags":["Task design"],"operationId":"getBatch","x-plumbline-scope":"batches:read","summary":"Get a batch; poll after setup until READY","parameters":[{"$ref":"#/components/parameters/BatchId"}],"responses":{"200":{"description":"The batch","content":{"application/json":{"schema":{"$ref":"#/components/schemas/Batch"}}}},"default":{"$ref":"#/components/responses/Error"}}}},"/data-collection/batches/{batch_id}/instructions":{"get":{"tags":["Task design"],"operationId":"listInstructions","x-plumbline-scope":"batches:read","summary":"List a batch's instructions","parameters":[{"$ref":"#/components/parameters/BatchIdNested"}],"responses":{"200":{"description":"The instructions, in order","content":{"application/json":{"schema":{"$ref":"#/components/schemas/InstructionList"}}}},"default":{"$ref":"#/components/responses/Error"}}},"post":{"tags":["Task design"],"operationId":"createInstructions","x-plumbline-scope":"batches:write","summary":"Add instructions to a batch","description":"Prolific's instruction types (multiple_choice, free_text, multiple_choice_with_free_text, file_upload)\nare accepted unchanged. Plumbline adds pairwise_preference.\n","parameters":[{"$ref":"#/components/parameters/BatchIdNested"}],"requestBody":{"required":true,"content":{"application/json":{"schema":{"$ref":"#/components/schemas/InstructionsCreate"},"example":{"instructions":[{"type":"pairwise_preference","description":"Which response is more helpful and accurate?","options":["response_a","response_b","tie"]},{"type":"free_text","description":"Briefly say why."}]}}}},"responses":{"201":{"description":"The created instructions","content":{"application/json":{"schema":{"$ref":"#/components/schemas/InstructionList"}}}},"default":{"$ref":"#/components/responses/Error"}}}},"/data-collection/batches/{id}/setup":{"post":{"tags":["Task design"],"operationId":"setupBatch","x-plumbline-scope":"batches:write","summary":"Freeze instructions and build tasks from the dataset","description":"Poll the batch until READY. Builds one task per item plus hidden test-question tasks.","parameters":[{"$ref":"#/components/parameters/BatchId"}],"requestBody":{"required":false,"content":{"application/json":{"schema":{"type":"object"}}}},"responses":{"202":{"description":"Setup started; the batch is PROCESSING","content":{"application/json":{"schema":{"$ref":"#/components/schemas/Batch"}}}},"200":{"description":"Setup finished at once; the batch is READY","content":{"application/json":{"schema":{"$ref":"#/components/schemas/Batch"}}}},"default":{"$ref":"#/components/responses/Error"}}}},"/data-collection/batches/{batch_id}/responses":{"get":{"tags":["Results"],"operationId":"listResponses","x-plumbline-scope":"results:read","summary":"Raw responses, one per rater per instruction, in Prolific's shape","parameters":[{"$ref":"#/components/parameters/BatchIdNested"},{"name":"include","in":"query","description":"`rater_profile` attaches the pseudonymous rater profile to each response.","schema":{"type":"string","enum":["rater_profile"]}},{"$ref":"#/components/parameters/Page"},{"$ref":"#/components/parameters/PerPage"}],"responses":{"200":{"description":"The responses","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ResponseList"}}}},"default":{"$ref":"#/components/responses/Error"}}}},"/data-collection/batches/{batch_id}/results":{"get":{"tags":["Results"],"operationId":"listResults","x-plumbline-scope":"results:read","summary":"The finished product, one record per item (Plumbline only)","description":"Votes, agreement, ties, quality flags, segments, DPO eligibility and the ai_judge block for each item.\n`?format=dpo_jsonl` or `?format=reward_jsonl` returns a training file (one JSON object per line) instead.\nBoth carry the standard `prompt`, `chosen`, `rejected` fields. dpo_jsonl lists only items with a clear\nwinner; reward_jsonl lists every finished item, with `chosen` and `rejected` null on ties and tied counts,\nplus `response_a`, `response_b`, `label`, `soft_label`, `weight` and, when the task has one, `rubric`.\n","parameters":[{"$ref":"#/components/parameters/BatchIdNested"},{"name":"format","in":"query","schema":{"type":"string","enum":["json","dpo_jsonl","reward_jsonl"]}},{"$ref":"#/components/parameters/Page"},{"$ref":"#/components/parameters/PerPage"}],"responses":{"200":{"description":"The results, or a JSONL training file","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ResultList"}},"application/x-ndjson":{"schema":{"type":"string"}}}},"default":{"$ref":"#/components/responses/Error"}}}},"/studies":{"get":{"tags":["Studies"],"operationId":"listStudies","x-plumbline-scope":"studies:read","summary":"List the workspace's studies","parameters":[{"name":"state","in":"query","description":"Filter by status.","schema":{"$ref":"#/components/schemas/StudyStatus"}},{"$ref":"#/components/parameters/Page"},{"$ref":"#/components/parameters/PerPage"}],"responses":{"200":{"description":"The studies","content":{"application/json":{"schema":{"$ref":"#/components/schemas/StudyList"}}}},"default":{"$ref":"#/components/responses/Error"}}},"post":{"tags":["Studies"],"operationId":"createStudy","x-plumbline-scope":"studies:write","summary":"Create the panel review request","description":"Prolific fields are kept; filters use Prolific's filter_id shape, mapped to panel profile fields.\nThe `plumbline_review` block sets panel and quality rules. The response carries `plumbline_quote`.\n`external_study_url`, `completion_codes` and `prolific_id_option` are accepted and echoed but not used,\nbecause tasks run inside Plumbline.\n","requestBody":{"required":true,"content":{"application/json":{"schema":{"$ref":"#/components/schemas/StudyCreate"},"example":{"name":"Helpfulness pairs, October","description":"Compare two AI answers.","data_collection_method":"AI_TASK_BUILDER_BATCH","data_collection_id":"bt_01J9X5B2MRD7F9H1K3M5P7R9TV","total_available_places":400,"estimated_completion_time":30,"reward":1200,"filters":[{"filter_id":"current-country-of-residence","selected_values":["US"]},{"filter_id":"fluent-languages","selected_values":["en"]}],"plumbline_review":{"raters_per_item":5,"panel_tier":"certified","quotas":[{"attribute":"age_band","split":{"18-34":0.3,"35-54":0.35,"55+":0.35}}],"gold_rate":0.05,"ai_judge":{"include":true}}}}}},"responses":{"201":{"description":"The new study, UNPUBLISHED, with its quote","content":{"application/json":{"schema":{"$ref":"#/components/schemas/Study"},"example":{"id":"st_01J9X6C0ZPE8G0J2L4N6Q8S0UW","status":"UNPUBLISHED","data_collection_id":"bt_01J9X5B2MRD7F9H1K3M5P7R9TV","plumbline_quote":{"participant_pay_cents":480000,"platform_fee_cents":158400,"fee_rate":0.33,"fee_is_placeholder":true,"currency":"USD"}}}}},"default":{"$ref":"#/components/responses/Error"}}}},"/studies/{id}":{"get":{"tags":["Studies"],"operationId":"getStudy","x-plumbline-scope":"studies:read","summary":"Get a study","description":"Status uses Prolific's values: UNPUBLISHED, SCHEDULED, PUBLISHING, ACTIVE, PAUSED, AWAITING REVIEW,\nCOMPLETED. `plumbline_progress` adds item-level progress.\n","parameters":[{"$ref":"#/components/parameters/StudyId"}],"responses":{"200":{"description":"The study","content":{"application/json":{"schema":{"$ref":"#/components/schemas/Study"},"example":{"id":"st_01J9X6C0ZPE8G0J2L4N6Q8S0UW","status":"ACTIVE","total_available_places":400,"places_taken":233,"plumbline_progress":{"items_total":2000,"items_complete":1164,"judgements_collected":6420,"flagged_for_review":41,"raters_removed_for_quality":3}}}}},"default":{"$ref":"#/components/responses/Error"}}}},"/studies/{id}/transition":{"post":{"tags":["Studies"],"operationId":"transitionStudy","x-plumbline-scope":"studies:publish","summary":"Publish, pause, resume or stop a study","description":"Prolific's actions and transitions: PUBLISH (UNPUBLISHED to PUBLISHING to ACTIVE), PAUSE (ACTIVE to PAUSED),\nSTART (PAUSED to ACTIVE), STOP (to AWAITING REVIEW, then COMPLETED). PUBLISH needs a READY batch, an\naccepted quote and, if the study turned the AI judge on in `plumbline_review.ai_judge`, the workspace's\nconsent. A study that never mentioned the judge publishes without consent; its items wait for consent\n(see `plumbline_warnings`). A transition that is not allowed\nanswers 400 with the Error body.\n","parameters":[{"$ref":"#/components/parameters/StudyId"}],"requestBody":{"required":true,"content":{"application/json":{"schema":{"$ref":"#/components/schemas/StudyTransition"},"example":{"action":"PUBLISH"}}}},"responses":{"200":{"description":"The study after the transition","content":{"application/json":{"schema":{"$ref":"#/components/schemas/Study"},"example":{"id":"st_01J9X6C0ZPE8G0J2L4N6Q8S0UW","status":"ACTIVE"}}}},"default":{"$ref":"#/components/responses/Error"}}}},"/studies/{id}/timeline":{"get":{"tags":["Studies"],"operationId":"getStudyTimeline","x-plumbline-scope":"studies:read","summary":"Hourly progress, forecast to finish and numbered events (Plumbline only)","description":"Feeds the console's charts and event list. The series starts at publishing (t 0), has one point at the end of each clock hour since, and ends with the study as it stands now (or when it finished). The forecast divides the judgements still needed by the recent pace (weighted over the last six hours); its range uses the 20th and 80th percentile of hourly pace. Events are numbered per study.","parameters":[{"$ref":"#/components/parameters/StudyId"},{"name":"interval","in":"query","schema":{"type":"string","enum":["hour"],"default":"hour"}}],"responses":{"200":{"description":"The timeline","content":{"application/json":{"schema":{"$ref":"#/components/schemas/Timeline"},"example":{"interval":"hour","series":[{"t":0,"items_complete":0,"judgements":0,"agreement":null},{"t":30,"items_complete":1164,"judgements":5820,"agreement":0.71}],"forecast":{"finish_in_hours":22,"range_hours":[18,28]},"events":[{"number":1,"t":0,"kind":"customer_action","title":"Published","detail":{"action":"publish","actor":"api_token:plb_abc123"},"cost_cents":null},{"number":2,"t":8,"kind":"detection","title":"Burst of fast submissions","detail":{"detector":"fast_burst","summary":"6 of 41 answers in the last hour came in under a third of the median time and were set aside."},"cost_cents":null}]}}}},"default":{"$ref":"#/components/responses/Error"}}}},"/plumbline/key":{"get":{"tags":["Keys"],"operationId":"getKey","summary":"The key that made this request (Plumbline only)","description":"What this key is and may do: its name, mode (`test` or `live`), workspace, scopes, expiry and rate\nlimits, so a script can check itself before it starts. Any key may call it. Keys are made and revoked\nin the console only; no key can make another.\n","x-plumbline-scope":"any","responses":{"200":{"description":"The key","headers":{"RateLimit-Limit":{"$ref":"#/components/headers/RateLimit-Limit"},"RateLimit-Remaining":{"$ref":"#/components/headers/RateLimit-Remaining"},"RateLimit-Reset":{"$ref":"#/components/headers/RateLimit-Reset"}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Key"},"example":{"name":"Training pipeline","prefix":"plb_live_7Gx2kQ","mode":"live","workspace_id":"ws_01J9X3Q8ZK7H2D4M6N8P0R2T4V","scopes":["datasets:read","batches:read","results:read","studies:read"],"expires_at":"2027-01-31T23:59:59Z","created_at":"2026-11-07T14:02:11Z","last_used_at":"2026-11-07T14:02:11Z","rate_limits":{"requests_per_minute":600,"workspace_requests_per_minute":3000}}}}},"default":{"$ref":"#/components/responses/Error"}}}},"/uploads/{token}":{"servers":[{"url":"https://plumbline.mikeporter.ai","description":"This server"},{"url":"https://api.plumbline.example"},{"url":"http://localhost:3000"}],"put":{"tags":["Uploads"],"operationId":"putUpload","summary":"Upload the raw dataset file to a signed URL","description":"The target of `upload_url`. The signed token in the path is the credential, so no Authorization header\nis needed. The body is the raw CSV or JSONL file. Ingest starts at once; poll the dataset until READY.\n","security":[],"parameters":[{"name":"token","in":"path","required":true,"schema":{"type":"string"}}],"requestBody":{"required":true,"content":{"application/octet-stream":{"schema":{"type":"string","format":"binary"}},"text/csv":{"schema":{"type":"string"}},"application/x-ndjson":{"schema":{"type":"string"}}}},"responses":{"200":{"description":"Received; the dataset is PROCESSING","content":{"application/json":{"schema":{"$ref":"#/components/schemas/Dataset"}}}},"default":{"$ref":"#/components/responses/Error"}}}}},"components":{"securitySchemes":{"TokenAuth":{"type":"apiKey","in":"header","name":"Authorization","description":"`Token <api-key>`, as Prolific clients send it. `Bearer <api-key>` is also accepted. A key starts\n`plb_live_` or `plb_test_` and ends with a checksum; it holds scopes (see each operation's\n`x-plumbline-scope`) and may have an expiry date.\n"}},"headers":{"RateLimit-Limit":{"description":"Requests allowed in the current one-minute window, for the tighter of the key's and the workspace's budgets.","schema":{"type":"string","pattern":"^[0-9]+$"}},"RateLimit-Remaining":{"description":"Requests left in the current window.","schema":{"type":"string","pattern":"^[0-9]+$"}},"RateLimit-Reset":{"description":"Seconds until the window ends and the count starts again.","schema":{"type":"string","pattern":"^[0-9]+$"}},"Retry-After":{"description":"On a 429, seconds to wait before trying again.","schema":{"type":"string","pattern":"^[0-9]+$"}}},"parameters":{"DatasetId":{"name":"id","in":"path","required":true,"schema":{"type":"string","pattern":"^ds_[0-9A-Z]{26}$"}},"BatchId":{"name":"id","in":"path","required":true,"schema":{"type":"string","pattern":"^bt_[0-9A-Z]{26}$"}},"BatchIdNested":{"name":"batch_id","in":"path","required":true,"schema":{"type":"string","pattern":"^bt_[0-9A-Z]{26}$"}},"StudyId":{"name":"id","in":"path","required":true,"schema":{"type":"string","pattern":"^st_[0-9A-Z]{26}$"}},"WorkspaceIdQuery":{"name":"workspace_id","in":"query","description":"Defaults to the token's workspace.","schema":{"type":"string"}},"Page":{"name":"page","in":"query","schema":{"type":"integer","minimum":1,"default":1}},"PerPage":{"name":"per_page","in":"query","schema":{"type":"integer","minimum":1,"maximum":500,"default":100}}},"responses":{"Error":{"description":"Any error (400, 401, 403, 404, 409, 422, 429, 500, 501), in Prolific's error body. `detail` says what\nwent wrong in words; for a refused key it starts with a reason a script can key on:\n`key_expired` (401, the key passed its expiry date), `key_revoked` (401, someone revoked it in the\nconsole). A 403 names the scope the key is missing. A 429 says which budget ran out (the key's or the\nworkspace's) and carries `Retry-After`.\n","headers":{"RateLimit-Limit":{"$ref":"#/components/headers/RateLimit-Limit"},"RateLimit-Remaining":{"$ref":"#/components/headers/RateLimit-Remaining"},"RateLimit-Reset":{"$ref":"#/components/headers/RateLimit-Reset"},"Retry-After":{"$ref":"#/components/headers/Retry-After"}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"},"example":{"error":{"status":401,"error_code":140101,"title":"Unauthorized","detail":"Send a valid API token in the Authorization header: Token <token>."}}}}}},"schemas":{"Error":{"type":"object","required":["error"],"properties":{"error":{"type":"object","required":["status","error_code","title"],"properties":{"status":{"type":"integer"},"error_code":{"type":"integer"},"title":{"type":"string"},"detail":{"description":"A message, or an object of field errors.","type":["string","object","null"]}}}}},"Key":{"type":"object","required":["name","prefix","mode","workspace_id","scopes","expires_at","created_at","last_used_at","rate_limits"],"properties":{"name":{"type":["string","null"]},"prefix":{"description":"The first characters of the key, as the console shows it.","type":"string"},"mode":{"type":"string","enum":["test","live"]},"workspace_id":{"type":"string"},"scopes":{"type":"array","items":{"type":"string","enum":["datasets:read","datasets:write","batches:read","batches:write","results:read","studies:read","studies:write","studies:publish"]}},"expires_at":{"description":"Null for a key with no expiry.","type":["string","null"],"format":"date-time"},"created_at":{"type":"string","format":"date-time"},"last_used_at":{"type":["string","null"],"format":"date-time"},"rate_limits":{"type":"object","required":["requests_per_minute","workspace_requests_per_minute"],"properties":{"requests_per_minute":{"type":"integer"},"workspace_requests_per_minute":{"type":"integer"}}}}},"Link":{"type":"object","required":["href"],"properties":{"href":{"type":["string","null"]},"title":{"type":"string"}}},"Links":{"type":"object","required":["next"],"properties":{"self":{"$ref":"#/components/schemas/Link"},"next":{"$ref":"#/components/schemas/Link"},"previous":{"$ref":"#/components/schemas/Link"},"last":{"$ref":"#/components/schemas/Link"}}},"Meta":{"type":"object","required":["count"],"properties":{"count":{"type":"integer","minimum":0}}},"DatasetStatus":{"type":"string","enum":["UNINITIALISED","PROCESSING","READY","ERROR"]},"DatasetCreate":{"type":"object","required":["name"],"properties":{"name":{"type":"string"},"workspace_id":{"type":"string","description":"Defaults to the token's workspace."},"schema":{"type":"string","description":"`pairwise_preference@v1` for pairwise items; omit for Prolific's generic datasets.","example":"pairwise_preference@v1"}}},"Dataset":{"type":"object","required":["id","status"],"properties":{"id":{"type":"string","pattern":"^ds_[0-9A-Z]{26}$"},"name":{"type":"string"},"workspace_id":{"type":"string"},"schema":{"type":["string","null"]},"status":{"$ref":"#/components/schemas/DatasetStatus"},"datapoint_count":{"type":"integer"},"validation":{"$ref":"#/components/schemas/DatasetValidation"},"categories":{"type":"object","description":"Rows per detected category.","additionalProperties":{"type":"integer"}},"created_at":{"type":"string","format":"date-time"}}},"DatasetValidation":{"type":"object","properties":{"rows_rejected":{"type":"integer"},"errors":{"type":"array","items":{"type":"object","required":["row","error"],"properties":{"row":{"type":"integer"},"error":{"type":"string"}}}}}},"DatasetList":{"type":"object","required":["results","meta","_links"],"properties":{"results":{"type":"array","items":{"$ref":"#/components/schemas/Dataset"}},"meta":{"$ref":"#/components/schemas/Meta"},"_links":{"$ref":"#/components/schemas/Links"}}},"UploadUrl":{"type":"object","required":["upload_url","expires_at","http_method","dataset_id"],"properties":{"upload_url":{"type":"string","format":"uri"},"expires_at":{"type":"string","format":"date-time"},"http_method":{"type":"string","enum":["PUT"]},"dataset_id":{"type":"string"}}},"BatchStatus":{"type":"string","enum":["UNINITIALISED","PROCESSING","READY","ERROR"]},"TaskDetails":{"type":"object","properties":{"task_name":{"type":"string"},"task_introduction":{"type":"string"},"task_steps":{"type":"string"}}},"PlumblineTask":{"type":"object","description":"Plumbline's native pairwise question. Prolific clients can leave it out.","properties":{"type":{"type":"string","enum":["pairwise_preference"]},"allow_tie":{"type":"boolean","default":true},"rubric":{"type":"array","description":"Criteria rated per response on a slider, e.g. helpful, accurate, safe.","items":{"type":"string"}},"rationale":{"type":"string","enum":["none","optional","required"],"default":"optional"},"counterbalance_order":{"type":"boolean","default":true}}},"BatchCreate":{"type":"object","required":["name","dataset_id"],"properties":{"name":{"type":"string"},"workspace_id":{"type":"string"},"dataset_id":{"type":"string"},"task_details":{"$ref":"#/components/schemas/TaskDetails"},"plumbline_task":{"$ref":"#/components/schemas/PlumblineTask"}}},"Batch":{"type":"object","required":["id","status"],"properties":{"id":{"type":"string","pattern":"^bt_[0-9A-Z]{26}$"},"name":{"type":"string"},"workspace_id":{"type":"string"},"dataset_id":{"type":["string","null"]},"status":{"$ref":"#/components/schemas/BatchStatus"},"task_details":{"$ref":"#/components/schemas/TaskDetails"},"plumbline_task":{"$ref":"#/components/schemas/PlumblineTask"},"total_task_count":{"type":"integer"},"created_at":{"type":"string","format":"date-time"}}},"BatchList":{"type":"object","required":["results","meta","_links"],"properties":{"results":{"type":"array","items":{"$ref":"#/components/schemas/Batch"}},"meta":{"$ref":"#/components/schemas/Meta"},"_links":{"$ref":"#/components/schemas/Links"}}},"InstructionType":{"type":"string","description":"Prolific's generic types plus Plumbline's pairwise_preference.","enum":["multiple_choice","free_text","multiple_choice_with_free_text","file_upload","pairwise_preference"]},"InstructionOption":{"description":"A choice. Pairwise options are the strings response_a, response_b and tie.","oneOf":[{"type":"string"},{"type":"object","required":["label","value"],"properties":{"label":{"type":"string"},"value":{"type":"string"}}}]},"InstructionInput":{"type":"object","required":["type","description"],"properties":{"type":{"$ref":"#/components/schemas/InstructionType"},"description":{"type":"string"},"options":{"type":"array","description":"Required for multiple_choice, multiple_choice_with_free_text and pairwise_preference.","items":{"$ref":"#/components/schemas/InstructionOption"}},"answer_limit":{"type":"integer","description":"Prolific's limit on choices for multiple_choice."}}},"InstructionsCreate":{"type":"object","required":["instructions"],"properties":{"instructions":{"type":"array","minItems":1,"items":{"$ref":"#/components/schemas/InstructionInput"}}}},"Instruction":{"type":"object","required":["id","type"],"properties":{"id":{"type":"string","pattern":"^in_[0-9A-Z]{26}$"},"type":{"$ref":"#/components/schemas/InstructionType"},"description":{"type":"string"},"options":{"type":"array","items":{"$ref":"#/components/schemas/InstructionOption"}},"position":{"type":"integer"}}},"InstructionList":{"type":"object","required":["instructions"],"properties":{"instructions":{"type":"array","items":{"$ref":"#/components/schemas/Instruction"}}}},"StudyStatus":{"type":"string","enum":["UNPUBLISHED","SCHEDULED","PUBLISHING","ACTIVE","PAUSED","AWAITING REVIEW","COMPLETED"]},"Filter":{"type":"object","required":["filter_id"],"properties":{"filter_id":{"type":"string","example":"current-country-of-residence"},"selected_values":{"type":"array","items":{"type":"string"}},"selected_range":{"type":"object","properties":{"lower":{"type":["number","null"]},"upper":{"type":["number","null"]}}}}},"Quota":{"type":"object","required":["attribute","split"],"properties":{"attribute":{"type":"string","example":"age_band"},"split":{"type":"object","description":"Target share per value; shares add up to 1.","additionalProperties":{"type":"number","minimum":0,"maximum":1}}}},"PlumblineReview":{"type":"object","description":"Panel and quality rules. Every field has a default.","properties":{"raters_per_item":{"type":"integer","minimum":1,"default":3,"description":"Raters on every item at first. With max_raters_per_item higher, a pairwise item gets more raters until one answer leads by stop_at_lead votes. By default items get 3 to 9 raters."},"max_raters_per_item":{"type":["integer","null"],"default":9,"description":"Adaptive rater counts: the most raters an item may get, at least raters_per_item. Left out, it is 9 (or raters_per_item if higher), unless the study names raters_per_item, in which case it is raters_per_item and every item gets exactly that many. Null also means exactly raters_per_item. Pairwise items only."},"stop_at_lead":{"type":"integer","minimum":1,"default":2,"description":"Votes the leading answer must be ahead of the next before an item stops getting raters. An item that reaches max_raters_per_item without this lead is marked contested."},"items_per_place":{"type":"integer","minimum":1,"default":40},"panel_tier":{"type":"string","enum":["any","verified","certified","expert"],"default":"any"},"quotas":{"type":"array","items":{"$ref":"#/components/schemas/Quota"}},"gold_rate":{"type":"number","minimum":0,"maximum":1,"default":0.05},"dpo_rule":{"type":"object","properties":{"min_majority_share":{"type":"number","default":0.6},"allow_tie_majority":{"type":"boolean","default":false}}},"ai_judge":{"type":"object","properties":{"include":{"type":"boolean","default":true}}},"sensitive":{"type":"boolean","default":false,"description":"The items may upset a rater (violence, self-harm, abuse). Such a study goes only to raters who opted in to upsetting content on the rater app; everyone else is screened out."}}},"PlumblineQuote":{"type":"object","required":["participant_pay_cents","platform_fee_cents","currency"],"properties":{"participant_pay_cents":{"type":"integer","description":"reward x total_available_places."},"platform_fee_cents":{"type":"integer"},"total_cents":{"type":"integer"},"fee_rate":{"type":"number"},"fee_is_placeholder":{"type":"boolean","description":"True while the platform fee is the 33% placeholder."},"currency":{"type":"string","example":"USD"},"accepted_at":{"type":["string","null"],"format":"date-time"},"raters_per_item":{"type":"object","description":"The fewest and most raters an item gets (equal without max_raters_per_item).","properties":{"minimum":{"type":"integer"},"maximum":{"type":"integer"}}},"places_needed":{"type":"object","description":"Estimate of the places the batch needs at the fewest and the most raters per item. Buy places for the most: a study stops taking places once every item has enough votes, so unneeded places are never taken or paid. Null without a batch.","properties":{"at_minimum":{"type":["integer","null"]},"at_maximum":{"type":["integer","null"]}}}}},"PlumblineProgress":{"type":"object","required":["items_total","items_complete","judgements_collected","flagged_for_review","raters_removed_for_quality"],"properties":{"items_total":{"type":"integer"},"items_complete":{"type":"integer"},"judgements_collected":{"type":"integer"},"flagged_for_review":{"type":"integer"},"raters_removed_for_quality":{"type":"integer"},"agreement":{"type":["number","null"]},"tie_rate":{"type":["number","null"]},"judge_spend_cents":{"type":"integer"},"judge_budget_cents":{"type":["integer","null"]}}},"StudyCreate":{"type":"object","required":["name"],"properties":{"name":{"type":"string"},"internal_name":{"type":"string"},"description":{"type":"string"},"data_collection_method":{"type":"string","enum":["AI_TASK_BUILDER_BATCH"]},"data_collection_id":{"type":"string","description":"The batch id (bt_...)."},"total_available_places":{"type":"integer","minimum":1},"estimated_completion_time":{"type":"integer","description":"Minutes."},"maximum_allowed_time":{"type":"integer","description":"Minutes."},"reward":{"type":"integer","description":"Per place, in cents."},"device_compatibility":{"type":"array","items":{"type":"string"}},"filters":{"type":"array","items":{"$ref":"#/components/schemas/Filter"}},"external_study_url":{"type":"string","description":"Accepted for compatibility; not used."},"completion_codes":{"type":"array","description":"Accepted for compatibility; not used.","items":{"type":"object"}},"prolific_id_option":{"type":"string","description":"Accepted for compatibility; not used."},"plumbline_review":{"$ref":"#/components/schemas/PlumblineReview"}}},"Study":{"type":"object","required":["id","status"],"properties":{"id":{"type":"string","pattern":"^st_[0-9A-Z]{26}$"},"name":{"type":"string"},"internal_name":{"type":["string","null"]},"description":{"type":["string","null"]},"status":{"$ref":"#/components/schemas/StudyStatus"},"data_collection_method":{"type":["string","null"]},"data_collection_id":{"type":["string","null"]},"total_available_places":{"type":["integer","null"]},"places_taken":{"type":"integer"},"estimated_completion_time":{"type":["integer","null"]},"maximum_allowed_time":{"type":["integer","null"]},"reward":{"type":["integer","null"]},"currency":{"type":"string"},"filters":{"type":"array","items":{"$ref":"#/components/schemas/Filter"}},"published_at":{"type":["string","null"],"format":"date-time"},"date_created":{"type":"string","format":"date-time"},"plumbline_review":{"$ref":"#/components/schemas/PlumblineReview"},"plumbline_quote":{"$ref":"#/components/schemas/PlumblineQuote"},"plumbline_progress":{"$ref":"#/components/schemas/PlumblineProgress"},"plumbline_warnings":{"type":"array","description":"Things the study asked for that Plumbline is not doing, so nothing is silently ignored. Empty when there are none. `filter_not_applied`: a Prolific filter the panel cannot answer yet is kept and echoed but does not narrow who is invited. `ai_judge_awaiting_consent`: the AI judge is on for the study but the workspace has not consented, so no item is sent to it; a study that never mentioned `plumbline_review.ai_judge` (an unchanged Prolific script) still publishes, and its items are scored once consent is given.","items":{"$ref":"#/components/schemas/PlumblineWarning"}}}},"PlumblineWarning":{"type":"object","required":["code","field","message"],"properties":{"code":{"type":"string","enum":["filter_not_applied","ai_judge_awaiting_consent"]},"field":{"type":"string"},"message":{"type":"string"}}},"StudyList":{"type":"object","required":["results","meta","_links"],"properties":{"results":{"type":"array","items":{"$ref":"#/components/schemas/Study"}},"meta":{"$ref":"#/components/schemas/Meta"},"_links":{"$ref":"#/components/schemas/Links"}}},"StudyTransition":{"type":"object","required":["action"],"properties":{"action":{"type":"string","enum":["PUBLISH","PAUSE","START","STOP"]}}},"Timeline":{"type":"object","required":["interval","series","forecast","events"],"properties":{"interval":{"type":"string","enum":["hour"]},"series":{"type":"array","items":{"type":"object","required":["t","items_complete"],"properties":{"t":{"type":"number","description":"Hours since the study was published."},"items_complete":{"type":"integer","description":"Items (test questions left out) with all the judgements they need."},"judgements":{"type":"integer","description":"Judgements on the items that count (not set aside by the quality checks); test questions left out."},"agreement":{"type":["number","null"],"description":"Mean majority share over the complete items; null before any is complete."}}}},"forecast":{"type":"object","properties":{"finish_in_hours":{"type":["number","null"]},"range_hours":{"type":["array","null"],"items":{"type":"number"},"minItems":2,"maxItems":2}}},"events":{"type":"array","items":{"type":"object","required":["t","kind","title"],"properties":{"number":{"type":"integer"},"t":{"type":"number"},"kind":{"type":"string","enum":["detection","customer_action","automatic_action"],"description":"detection (found by a detector: low agreement, a slow quota cell, a burst of fast submissions), customer_action (your team or API token did it) or automatic_action (Plumbline did it, e.g. a rater removed for quality or the study completing)."},"title":{"type":"string"},"detail":{"type":"object"},"cost_cents":{"type":["integer","null"]}}}}}},"RaterProfile":{"type":"object","description":"Pseudonymous and coarse. Never contact details.","properties":{"age_band":{"type":"string"},"country":{"type":"string"},"languages":{"type":"array","items":{"type":"string"}},"education":{"type":"string"},"occupation":{"type":"string"},"tier":{"type":"string"},"tasks_completed":{"type":"integer"},"gold_accuracy":{"type":["number","null"]},"level":{"type":"string","description":"The rater's verification level. Set from recorded checks, never by hand.","enum":["joined","real_and_unique","identity_verified","proven_rater","verified_expert"]},"level_since":{"type":"string","format":"date-time","description":"When the rater reached this level."},"identity_checked_at":{"type":["string","null"],"format":"date","description":"Date of the identity check in force, or null when there is none."},"evidence":{"type":"object","description":"What backs each profile field; the strongest evidence on record for it. A professional link adds a key named after its provider (linkedin, github, x) whose value is `linked` (the rater signed in with that account and its name matched their ID) or `self_reported` (a pasted handle); the handle, name and URL are never shown.","additionalProperties":{"type":"string","enum":["document","registry","tested","linked","consistent","self_reported"]}}}},"ResponseRecord":{"type":"object","required":["id","batch_id","participant_id","submission_id","task_id","response","created_at"],"properties":{"id":{"type":"string","pattern":"^rs_[0-9A-Z]{26}$"},"batch_id":{"type":"string"},"participant_id":{"type":"string","description":"rtr_ plus 12 hex characters, salted per workspace.","pattern":"^rtr_[0-9a-f]{12}$"},"submission_id":{"type":"string"},"task_id":{"type":"string"},"response":{"type":"object","required":["instruction_id","type","answer"],"properties":{"instruction_id":{"type":"string"},"type":{"$ref":"#/components/schemas/InstructionType"},"answer":{"type":"array","items":{"type":"object","required":["value"],"properties":{"value":{"type":"string"}}}}}},"rater_profile":{"$ref":"#/components/schemas/RaterProfile"},"created_at":{"type":"string","format":"date-time"}}},"ResponseList":{"type":"object","required":["results","meta","_links"],"properties":{"results":{"type":"array","items":{"$ref":"#/components/schemas/ResponseRecord"}},"meta":{"$ref":"#/components/schemas/Meta"},"_links":{"$ref":"#/components/schemas/Links"}}},"PairwiseAnswer":{"type":"string","enum":["response_a","response_b","tie"]},"Votes":{"type":"object","properties":{"response_a":{"type":"number"},"response_b":{"type":"number"},"tie":{"type":"number"}}},"AiJudgeBlock":{"type":"object","description":"Model output, kept apart from the human fields, under a Plumbline-owned version string.\n`status` is scored, pending, awaiting_consent (the workspace has not consented), skipped_budget\n(a spend cap would be crossed) or off (the study opted out).\n","required":["status"],"properties":{"status":{"type":"string","enum":["scored","pending","awaiting_consent","skipped_budget","off"]},"judge_version":{"type":"string","example":"plumbline-judge-2026.10"},"rubric":{"type":"string","example":"pairwise-helpful@v1"},"p":{"$ref":"#/components/schemas/Votes"},"confidence":{"type":["number","null"]},"calibrated":{"type":"boolean"},"order_check":{"type":["string","null"],"enum":["consistent","inconsistent",null]},"agrees_with_majority":{"type":["boolean","null"]}}},"ItemResult":{"type":"object","required":["item_id","task_id","status","human"],"properties":{"item_id":{"type":["string","null"],"description":"The item_id from the uploaded row."},"task_id":{"type":"string"},"category":{"type":["string","null"]},"status":{"type":"string","enum":["pending","in_progress","complete"]},"simulated":{"type":"boolean","description":"True only in sandbox workspaces, for simulated crowds. Never a human label."},"human":{"type":"object","properties":{"raters":{"type":"integer"},"votes":{"$ref":"#/components/schemas/Votes"},"majority":{"description":"Null on a tied vote count.","type":["string","null"],"enum":["response_a","response_b","tie",null]},"agreement":{"type":["number","null"]},"soft_label":{"$ref":"#/components/schemas/Votes"},"rubric_means":{"type":"object","additionalProperties":{"type":"object","properties":{"response_a":{"type":["number","null"]},"response_b":{"type":["number","null"]}}}},"order_shown":{"type":"object","properties":{"a_first":{"type":"integer"},"b_first":{"type":"integer"}}},"raters_added":{"type":"integer","description":"Raters the stopping rule added past raters_per_item. Only when items can get more raters."},"contested":{"type":"boolean","description":"Complete at max_raters_per_item without a lead of stop_at_lead votes: people split on this item. Only when items can get more raters."},"instructions":{"type":"array","description":"Answers to the batch's instructions other than the pairwise question (Prolific's generic\ntasks): choice instructions as counts and shares per value, free text as the answers given.\n","items":{"$ref":"#/components/schemas/InstructionAnswers"}}}},"segments":{"type":"object","description":"Majority per segment value, e.g. {\"age_band\": {\"18-34\": \"response_a\"}}.","additionalProperties":{"type":"object","additionalProperties":{"type":["string","null"]}}},"quality":{"type":"object","properties":{"gold_pass_rate":{"type":["number","null"]},"flags":{"type":"array","items":{"type":"string"}}}},"ai_judge":{"$ref":"#/components/schemas/AiJudgeBlock"},"dpo":{"type":"object","properties":{"eligible":{"type":"boolean"},"chosen":{"type":["string","null"]},"rejected":{"type":["string","null"]},"rule":{"type":"string","example":"majority >= 0.6 and majority != tie"}}},"judgements":{"type":"array","description":"Every valid judgement. `answer` and `order` are given for pairwise tasks.","items":{"type":"object","required":["rater_id"],"properties":{"rater_id":{"type":"string"},"answer":{"$ref":"#/components/schemas/PairwiseAnswer"},"answers":{"type":"object","description":"Answers to the batch's other instructions, keyed by instruction id.","additionalProperties":{"type":"array","items":{"type":"string"}}},"order":{"type":"string","enum":["a_first","b_first"]},"rationale":{"type":["string","null"]},"seconds":{"type":["number","null"]},"rubric":{"type":"object"},"rater_profile":{"$ref":"#/components/schemas/RaterProfile"}}}}}},"InstructionAnswers":{"type":"object","required":["instruction_id","type","responses"],"properties":{"instruction_id":{"type":"string"},"type":{"$ref":"#/components/schemas/InstructionType"},"responses":{"type":"integer","description":"Valid judgements that answered this instruction."},"counts":{"type":"object","additionalProperties":{"type":"integer"}},"distribution":{"type":"object","additionalProperties":{"type":"number"}},"values":{"type":"array","items":{"type":"string"}}}},"ResultList":{"type":"object","required":["results","meta","_links"],"properties":{"results":{"type":"array","items":{"$ref":"#/components/schemas/ItemResult"}},"meta":{"type":"object","required":["count"],"properties":{"count":{"type":"integer"},"complete":{"type":"integer"}}},"summary":{"type":"object","properties":{"agreement_mean":{"type":["number","null"]},"tie_rate":{"type":["number","null"]},"dpo_eligible":{"type":"integer"},"contested":{"type":"integer","description":"Complete items marked contested (0 when items get a fixed count)."}}},"_links":{"$ref":"#/components/schemas/Links"}}}}}}