feat: add the openapi docs

This commit is contained in:
2026-08-16 16:43:21 +07:00
parent 92d4da14ed
commit a235040be7
27 changed files with 3558 additions and 2837 deletions
+27
View File
@@ -0,0 +1,27 @@
# Local-only Cloudflare Worker bindings. Copy to .dev.vars and replace each value.
# Secrets must never be committed.
# AUTH_KV, FOLDER_CACHE, and MPU are configured bindings in wrangler.jsonc, not .dev.vars values.
# S3 Signature V4 credentials held by CLI clients or your BFF.
ACCESS_KEY=replace-with-access-key-id
SECRET_KEY=replace-with-long-random-secret
REGION=auto
# Google OAuth credentials with Drive API access.
GOOGLE_CLIENT_ID=replace-with-google-oauth-client-id
GOOGLE_CLIENT_SECRET=replace-with-google-oauth-client-secret
GOOGLE_REFRESH_TOKEN=replace-with-google-refresh-token
# Comma-separated, exact bucket names. Unset denies every bucket.
ALLOWED_BUCKETS=assets
# Optional comma-separated subset of ALLOWED_BUCKETS that permits unsigned GET/HEAD.
PUBLIC_READ_BUCKETS=
# Durable Object multipart uploads and ETag result behavior.
ALLOW_MULTIPART=true
ETAG_STYLE=md5
# Browser CORS: comma-separated exact origins or *. Leave empty to disable CORS.
CORS_ALLOWED_ORIGINS=http://localhost:5173
# Set false to disable /docs and /openapi.yaml.
ENABLE_DOCS=true
+15 -2
View File
@@ -3,6 +3,10 @@ I.R.I.S. (Integrated Reliable Interop Storage)
Use Cloudflare Workers to turn your Google Drive into S3 object storage at no extra cost.
## API documentation
With `ENABLE_DOCS` left enabled (the default), each deployment serves an interactive API reference at `/docs` and the raw OpenAPI 3.1 document at `/openapi.yaml`. Frontend-specific setup, authentication details, limitations, and runnable examples live in [`docs/`](./docs/README.md).
## About
This is a Workers script that converts the Google Drive API into an S3-compatible API. Turn your Google Drive into object storage at no extra cost!
@@ -74,6 +78,8 @@ https://developers.cloudflare.com/workers/configuration/secrets/#via-the-dashboa
| `GOOGLE_CLIENT_ID`, `GOOGLE_CLIENT_SECRET`, `GOOGLE_REFRESH_TOKEN` | Google API credentials obtained from rclone. |
| `ALLOWED_BUCKETS` | Set the buckets allowed, separated by `,`. A directory with the bucket name will be created directly under Google Drive. |
| `PUBLIC_READ_BUCKETS` | *(Optional)* Buckets that allow unauthenticated GET/HEAD access without signature, separated by `,`. Write operations (PUT/POST/DELETE) still require authentication. Must be a subset of `ALLOWED_BUCKETS`. |
| `CORS_ALLOWED_ORIGINS` | *(Optional)* Comma-separated exact browser origins, or `*`. Unset emits no CORS headers. |
| `ENABLE_DOCS` | *(Optional)* Set to `false` to disable `/docs` and `/openapi.yaml`; enabled by default. |
### 4. Enable Multipart Uploads
@@ -95,5 +101,12 @@ Google Drive's free tier has 15 GB total storage, and Google applies a 750 GB da
### 5. CORS Configuration
If you need to configure CORS, set up your own domain for Workers and use Cloudflare's Response Header Transform Rules to add the necessary headers.
https://developers.cloudflare.com/rules/transform/response-header-modification/
I.R.I.S. provides native, deny-by-default CORS handling. Set `CORS_ALLOWED_ORIGINS` to a comma-separated list of exact origins:
```ini
CORS_ALLOWED_ORIGINS=https://app.example.com,http://localhost:5173
```
Use `*` only for a public, credential-free integration. Leave the value unset to emit no CORS headers, preserving CLI-only behavior. Allowed origins receive preflight support for S3 methods and the browser-readable response headers `ETag`, `Content-Range`, `Content-Length`, `Last-Modified`, `Accept-Ranges`, and `x-amz-request-id`. Server-to-server clients do not need CORS: requests without an `Origin` retain their normal S3 response and receive no `Access-Control-*` headers. Exact origin allow-lists add the harmless `Vary: Origin` response header for cache correctness.
For browser uploads, keep `SECRET_KEY` in a BFF and give the browser short-lived presigned URLs. See the [frontend integration guide](./docs/integration-guide.md).
+1 -1
View File
@@ -7,7 +7,7 @@
},
"files": {
"ignoreUnknown": true,
"includes": ["**", "!node_modules", "!.next", "!dist", "!build", "!src/app/globals.css"]
"includes": ["**", "!node_modules", "!.next", "!dist", "!build", "!src/app/globals.css", "!worker-configuration.d.ts"]
},
"formatter": {
"lineWidth": 320,
+29
View File
@@ -0,0 +1,29 @@
# I.R.I.S. frontend integration docs
I.R.I.S. exposes a focused S3-compatible API on top of Google Drive. The live API reference is available from every enabled Worker deployment at [`/docs`](/docs), with the source OpenAPI document at [`/openapi.yaml`](/openapi.yaml).
## Start here
1. Configure the Worker with `CORS_ALLOWED_ORIGINS` for every browser origin that will upload or download objects.
2. Keep `SECRET_KEY` in a backend-for-frontend (BFF), not in browser code.
3. Let the BFF issue short-lived presigned URLs for a constrained bucket/key/method.
4. Upload or download through the presigned URL from the browser.
| Document | Purpose |
| --- | --- |
| [Integration guide](./integration-guide.md) | Browser upload, multipart upload, listing, CORS, and recommended architecture. |
| [Authentication](./authentication.md) | The precise AWS Signature V4 behavior enforced by I.R.I.S. |
| [Limitations](./limitations.md) | Unsupported S3 features and Google Drive operational limits. |
| [`examples/bff-presign.ts`](./examples/bff-presign.ts) | A Workers BFF endpoint that creates short-lived PUT URLs. |
| [`examples/browser-upload.ts`](./examples/browser-upload.ts) | Single object upload from a browser. |
| [`examples/browser-multipart.ts`](./examples/browser-multipart.ts) | Strictly sequential browser multipart upload. |
## Browser security model
```text
Browser ── POST /presign ──> BFF (holds SECRET_KEY)
│ │
└──── presigned S3 request ────┴──> I.R.I.S. Worker ──> Google Drive
```
The browser never receives `SECRET_KEY`. Presigned URLs must be short lived and scoped to the single object operation the browser needs.
+39
View File
@@ -0,0 +1,39 @@
# Authentication and Signature V4
I.R.I.S. accepts AWS Signature Version 4 in either form:
- an `Authorization: AWS4-HMAC-SHA256 …` header with `x-amz-date`; or
- a presigned request with `X-Amz-Algorithm`, `X-Amz-Credential`, `X-Amz-Date`, `X-Amz-SignedHeaders`, and `X-Amz-Signature` query parameters.
The `Credential` access-key ID must equal the Worker's `ACCESS_KEY`; the signing key is derived from `SECRET_KEY`, `REGION`, service `s3`, and the `YYYYMMDD` request date.
## Presigned URL expiry
Query-authenticated requests require `X-Amz-Expires` from 1 through 604800 seconds (seven days). The Worker returns `403 AccessDenied` with `Request has expired` when it is missing, invalid, out of range, or its signed timestamp plus expiry is in the past.
BFFs should normally use a much shorter period such as five minutes.
## Header-authenticated requests
The Worker requires a valid `x-amz-date` and rejects a request when the server time differs from it by more than 15 minutes. This returns `403 RequestTimeTooSkewed`.
## Canonical request behavior
The canonical request is built from:
1. method;
2. URL path;
3. sorted query string (excluding `X-Amz-Signature` for presigned URLs);
4. the signed-header list and normalized values;
5. the signed-header list; and
6. `x-amz-content-sha256`, defaulting to `UNSIGNED-PAYLOAD`.
I.R.I.S. deliberately canonicalizes a signed `accept-encoding` header to `identity`. Cloudflare can rewrite the received value at the edge; S3 SDKs that sign this header use `identity` for this reason. Browser code must not sign `accept-encoding` because browser networking controls it.
Payload hashes are **not** verified. Browser and BFF clients should use `x-amz-content-sha256: UNSIGNED-PAYLOAD`; this is a deliberate streaming limitation, not an integrity guarantee.
## Public-read buckets
Buckets listed in `PUBLIC_READ_BUCKETS` permit unsigned `GET` and `HEAD` requests. All write operations still require valid Signature V4 authentication. `PUBLIC_READ_BUCKETS` must be a subset of `ALLOWED_BUCKETS`.
For a tested signing reference, see the `signed()` helper in [`test/s3.test.ts`](../test/s3.test.ts).
+95
View File
@@ -0,0 +1,95 @@
import { AbortMultipartUploadCommand, CompleteMultipartUploadCommand, CreateMultipartUploadCommand, PutObjectCommand, S3Client, UploadPartCommand } from "@aws-sdk/client-s3";
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
interface Env {
IRIS_ENDPOINT: string;
IRIS_ACCESS_KEY: string;
IRIS_SECRET_KEY: string;
IRIS_REGION: string;
IRIS_BUCKET: string;
}
interface PresignRequest {
key: string;
contentType?: string;
operation?: "put" | "createMultipart" | "uploadPart" | "completeMultipart" | "abortMultipart";
uploadId?: string;
partNumber?: number;
parts?: Array<{ ETag: string; PartNumber: number }>;
}
function escapeXml(value: string): string {
return value.replace(/&/g, "&amp;").replace(/</g, "&lt;").replace(/>/g, "&gt;").replace(/"/g, "&quot;").replace(/'/g, "&apos;");
}
function completionXml(parts: Array<{ ETag: string; PartNumber: number }> | undefined): string | null {
if (!parts?.length || parts.some((part, index) => !Number.isInteger(part.PartNumber) || part.PartNumber !== index + 1 || typeof part.ETag !== "string" || part.ETag.length === 0)) return null;
return `<CompleteMultipartUpload>${parts.map((part) => `<Part><PartNumber>${part.PartNumber}</PartNumber><ETag>${escapeXml(part.ETag)}</ETag></Part>`).join("")}</CompleteMultipartUpload>`;
}
function json(data: unknown, status = 200): Response {
return Response.json(data, { status, headers: { "Cache-Control": "no-store" } });
}
function s3(env: Env): S3Client {
return new S3Client({
endpoint: env.IRIS_ENDPOINT,
region: env.IRIS_REGION,
forcePathStyle: true,
credentials: { accessKeyId: env.IRIS_ACCESS_KEY, secretAccessKey: env.IRIS_SECRET_KEY },
});
}
function validKey(key: unknown): key is string {
return typeof key === "string" && key.length > 0 && key.length <= 1024 && !key.startsWith("/") && !key.split("/").includes("..");
}
export default {
async fetch(request: Request, env: Env): Promise<Response> {
if (request.method !== "POST" || new URL(request.url).pathname !== "/presign") return new Response("Not found", { status: 404 });
let input: PresignRequest;
try {
input = await request.json<PresignRequest>();
} catch {
return json({ error: "Expected JSON body" }, 400);
}
if (!validKey(input.key)) return json({ error: "Invalid object key" }, 400);
const client = s3(env);
const operation = input.operation ?? "put";
const expiresIn = 5 * 60;
if (operation === "put") {
const contentType = input.contentType || "application/octet-stream";
const url = await getSignedUrl(client, new PutObjectCommand({ Bucket: env.IRIS_BUCKET, Key: input.key, ContentType: contentType }), { expiresIn });
return json({ url, method: "PUT", headers: { "Content-Type": contentType }, expiresIn });
}
if (operation === "createMultipart") {
const contentType = input.contentType || "application/octet-stream";
const url = await getSignedUrl(client, new CreateMultipartUploadCommand({ Bucket: env.IRIS_BUCKET, Key: input.key, ContentType: contentType }), { expiresIn });
return json({ url, method: "POST", headers: { "Content-Type": contentType }, expiresIn });
}
if (!input.uploadId) return json({ error: "uploadId is required" }, 400);
if (operation === "uploadPart") {
const partNumber = input.partNumber;
if (!Number.isInteger(partNumber) || partNumber === undefined || partNumber < 1 || partNumber > 10_000) return json({ error: "partNumber must be 1 through 10000" }, 400);
const url = await getSignedUrl(client, new UploadPartCommand({ Bucket: env.IRIS_BUCKET, Key: input.key, UploadId: input.uploadId, PartNumber: partNumber }), { expiresIn });
return json({ url, method: "PUT", expiresIn });
}
if (operation === "completeMultipart") {
const body = completionXml(input.parts);
if (!body) return json({ error: "parts must have sequential PartNumber values beginning at 1 and non-empty ETag values" }, 400);
const url = await getSignedUrl(client, new CompleteMultipartUploadCommand({ Bucket: env.IRIS_BUCKET, Key: input.key, UploadId: input.uploadId, MultipartUpload: { Parts: input.parts } }), { expiresIn });
return json({ url, method: "POST", headers: { "Content-Type": "application/xml" }, body, expiresIn });
}
if (operation === "abortMultipart") {
const url = await getSignedUrl(client, new AbortMultipartUploadCommand({ Bucket: env.IRIS_BUCKET, Key: input.key, UploadId: input.uploadId }), { expiresIn });
return json({ url, method: "DELETE", expiresIn });
}
return json({ error: "Unsupported operation" }, 400);
},
} satisfies ExportedHandler<Env>;
+48
View File
@@ -0,0 +1,48 @@
interface PresignResponse {
url: string;
method: string;
headers?: Record<string, string>;
body?: string;
}
async function presign(body: Record<string, unknown>): Promise<PresignResponse> {
const response = await fetch("/presign", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(body),
});
if (!response.ok) throw new Error(await response.text());
return response.json<PresignResponse>();
}
function uploadIdFromXml(xml: string): string {
const value = /<UploadId>([^<]+)<\/UploadId>/.exec(xml)?.[1];
if (!value) throw new Error("CreateMultipartUpload did not return UploadId");
return value;
}
/** Uploads parts in order. I.R.I.S. does not allow parallel or out-of-order part uploads. */
export async function uploadLargeFile(file: File, key: string, partSize = 16 * 1024 * 1024): Promise<void> {
const contentType = file.type || "application/octet-stream";
const create = await presign({ key, contentType, operation: "createMultipart" });
const createResponse = await fetch(create.url, { method: create.method, headers: create.headers });
if (!createResponse.ok) throw new Error(await createResponse.text());
const uploadId = uploadIdFromXml(await createResponse.text());
const parts: Array<{ ETag: string; PartNumber: number }> = [];
for (let index = 0, offset = 0; offset < file.size; index++, offset += partSize) {
const partNumber = index + 1;
const body = file.slice(offset, Math.min(offset + partSize, file.size));
const signed = await presign({ key, uploadId, partNumber, operation: "uploadPart" });
const response = await fetch(signed.url, { method: signed.method, body });
if (!response.ok) throw new Error(`Part ${partNumber} failed: ${await response.text()}`);
const etag = response.headers.get("ETag");
if (!etag) throw new Error(`Part ${partNumber} did not return ETag; check CORS_ALLOWED_ORIGINS`);
parts.push({ ETag: etag, PartNumber: partNumber });
}
const complete = await presign({ key, uploadId, parts, operation: "completeMultipart" });
if (!complete.body) throw new Error("BFF did not return the completion XML body");
const completeResponse = await fetch(complete.url, { method: complete.method, headers: complete.headers, body: complete.body });
if (!completeResponse.ok) throw new Error(await completeResponse.text());
}
+29
View File
@@ -0,0 +1,29 @@
interface PresignedPut {
url: string;
method: "PUT";
headers: { "Content-Type": string };
expiresIn: number;
}
async function getPresignedPut(key: string, contentType: string): Promise<PresignedPut> {
const response = await fetch("/presign", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ key, contentType, operation: "put" }),
});
if (!response.ok) throw new Error(await response.text());
return response.json<PresignedPut>();
}
export async function uploadFile(file: File, key: string): Promise<string | null> {
const signed = await getPresignedPut(key, file.type || "application/octet-stream");
const response = await fetch(signed.url, {
method: signed.method,
headers: signed.headers,
body: file,
});
if (!response.ok) throw new Error(`Upload failed: ${await response.text()}`);
// Requires CORS_ALLOWED_ORIGINS to include this page's exact origin.
return response.headers.get("ETag");
}
+98
View File
@@ -0,0 +1,98 @@
# Frontend integration guide
## Architecture
Use a backend-for-frontend (BFF) as the trust boundary:
```text
Browser → BFF (holds ACCESS_KEY + SECRET_KEY, issues presigned URLs) → I.R.I.S. Worker → Google Drive
```
Never put `SECRET_KEY` in browser code, a static site, or a public environment variable. The browser receives only a short-lived presigned URL for one method and object key.
## Enable browser access
Configure the Worker with an explicit allow-list of web origins:
```ini
CORS_ALLOWED_ORIGINS=https://app.example.com,http://localhost:5173
```
Set it to `*` only for a genuinely public, credential-free integration. Leaving it unset emits no CORS headers. I.R.I.S. exposes `ETag`, `Content-Range`, `Content-Length`, `Last-Modified`, `Accept-Ranges`, and `x-amz-request-id`, so browser code can inspect the headers that form the effective S3 response payload.
### Server-to-server compatibility
CORS does not restrict non-browser clients. `Access-Control-*` headers are emitted only when a request has an allowed `Origin`; a CLI, Dokploy, rclone, aws-cli, or other service request with no `Origin` keeps its normal S3 status and body. With an exact origin allow-list, responses also include `Vary: Origin`, including requests without an `Origin`, so shared caches cannot replay a no-origin object response to a browser. S3 clients ignore this cache-control metadata.
An `OPTIONS` request with no `Origin`, or a preflight from a disallowed origin, receives the same bare `204` as a CORS-disabled Worker. A disallowed `Origin` never changes the underlying S3 result—it simply receives no `Access-Control-*` headers.
## Presign in the BFF
Use `@aws-sdk/s3-request-presigner` and send a limited expiry. I.R.I.S. validates `X-Amz-Expires` and rejects URLs after their signing time plus that value.
```ts
import { PutObjectCommand, S3Client } from "@aws-sdk/client-s3";
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
const client = new S3Client({
endpoint: env.IRIS_ENDPOINT,
region: env.IRIS_REGION,
credentials: {
accessKeyId: env.IRIS_ACCESS_KEY,
secretAccessKey: env.IRIS_SECRET_KEY,
},
forcePathStyle: true,
});
const url = await getSignedUrl(
client,
new PutObjectCommand({
Bucket: "assets",
Key: "uploads/avatar.png",
ContentType: "image/png",
}),
{ expiresIn: 300 },
);
```
Keep the `ContentType` in the command and send precisely that same `Content-Type` from the browser. It becomes the Drive `mimeType`. A non-simple content type causes a preflight request, which is why the Worker CORS configuration is necessary.
## Upload one object
Use `Blob`, `File`, `ArrayBuffer`, or a typed array as the body. This lets the browser provide a known body length. Do **not** use a `ReadableStream` unless the BFF signs `x-amz-decoded-content-length` and the browser sends it: multipart part uploads require `Content-Length` or `x-amz-decoded-content-length`.
```ts
const response = await fetch(presignedUrl, {
method: "PUT",
headers: { "Content-Type": file.type || "application/octet-stream" },
body: file,
});
if (!response.ok) throw new Error(await response.text());
const etag = response.headers.get("ETag");
```
`ETag` is readable only when the request `Origin` is configured in `CORS_ALLOWED_ORIGINS`.
## Multipart upload
Cloudflare request limits mean a single PUT is best kept below roughly 100 MB. For larger files, have the BFF initiate the upload, issue a presigned URL for each part, and complete the upload after the browser returns all ETags.
Use 16 MB parts as a practical baseline. Upload parts **strictly sequentially starting at 1**. I.R.I.S. waits up to 20 seconds for an earlier part; out-of-order work then receives `503 SlowDown`. Requests more than 64 part numbers ahead are rejected immediately.
1. BFF signs `POST /bucket/key?uploads` and starts the multipart upload.
2. BFF issues a signed `PUT` URL for part 1; browser uploads it and records the `ETag`.
3. Repeat one part at a time for parts 2 through $n$.
4. BFF signs `POST /bucket/key?uploadId=…`; browser sends the `CompleteMultipartUpload` XML body containing the sequential part numbers and ETags.
See [`examples/browser-multipart.ts`](./examples/browser-multipart.ts) for a complete browser-side loop.
## Build a file browser
List with `prefix` and `delimiter=/`:
```text
GET /assets?prefix=photos/&delimiter=/
```
Files appear in `Contents`; immediate child folders appear as `CommonPrefixes`. Google Drive directories are physical folders, not zero-byte marker objects. Read [limitations](./limitations.md) before designing pagination or rename/move features.
+35
View File
@@ -0,0 +1,35 @@
# Limitations
I.R.I.S. is a narrow S3 compatibility layer. Design frontend features around the behavior below.
## Listing is not pageable
`max-keys`, `continuation-token`, and `marker` are ignored. I.R.I.S. scans at most 5000 Google Drive nodes and then sets `IsTruncated=true`; it does not provide a continuation token, so additional results cannot be requested. The XML value `MaxKeys=1000` is a fixed compatibility value, not the real scan limit.
Use narrow prefixes and `delimiter=/` for a file browser. Do not build an infinite-scrolling full-drive browser.
## No server-side rename or move
`CopyObject` and `UploadPartCopy` return `501 NotImplemented`. There is no server-side rename or move because both require copying. Implement rename/move as download, re-upload under the target key, then delete the original—subject to bandwidth, time, and error handling.
## Unsupported S3 features
- `ListBuckets`, `CreateBucket`, and `DeleteBucket`
- batch `DeleteObjects`
- object versioning, ACLs, object tags, lifecycle rules, and `x-amz-meta-*` user metadata
- CopyObject and UploadPartCopy
- discovering active multipart uploads: `ListMultipartUploads` always reports an empty list
Bucket `?acl`, `?versioning`, and `?location` requests are currently treated as regular bucket listing requests rather than errors. Do not rely on this accidental compatibility.
## Directories and objects
Directories are physical Google Drive folders created from object-key path segments. I.R.I.S. does not create zero-byte directory marker objects. An empty prefix can therefore exist as a Drive folder without an S3 marker object.
## Multipart behavior
Parts must be uploaded strictly in consecutive order beginning at part 1. Parts are immutable after they have committed. An incomplete upload expires after 24 hours. `ETAG_STYLE=md5` returns Google Drive's file MD5 for completed multipart objects; `ETAG_STYLE=multipart` is available only for clients that require an S3-style composite ETag.
## Quotas and plan limits
Google Drive's free tier has 15 GB total storage and a 750 GB/day upload limit. Cloudflare applies request-size limits before a request reaches the Worker; use multipart uploads for large browser uploads.
+391
View File
@@ -0,0 +1,391 @@
openapi: 3.1.0
info:
title: I.R.I.S. Google Drive S3 API
version: 1.0.0
description: |
An intentionally small S3-compatible object-storage API backed by Google Drive.
Authenticate every non-public request with AWS Signature Version 4. Browser clients
should receive short-lived presigned URLs from a backend-for-frontend, never credentials.
servers:
- url: https://{worker-host}
variables:
worker-host:
default: your-worker.example.workers.dev
security:
- sigv4: []
tags:
- name: Objects
- name: Buckets
- name: Multipart uploads
paths:
/{bucket}:
parameters:
- $ref: '#/components/parameters/Bucket'
get:
tags: [Buckets]
operationId: listObjects
summary: List objects in a bucket
description: List objects recursively, or use `delimiter=/` to return immediate folders as `CommonPrefixes`.
parameters:
- $ref: '#/components/parameters/Prefix'
- $ref: '#/components/parameters/Delimiter'
responses:
'200':
description: Object listing.
content:
application/xml:
schema: { $ref: '#/components/schemas/ListBucketResult' }
'403': { $ref: '#/components/responses/AccessDenied' }
'405': { $ref: '#/components/responses/MethodNotAllowed' }
head:
tags: [Buckets]
operationId: headBucket
summary: Check bucket access
responses:
'200': { description: The bucket is configured and accessible. }
'403': { $ref: '#/components/responses/AccessDenied' }
'405': { $ref: '#/components/responses/MethodNotAllowed' }
/{bucket}?uploads:
parameters:
- $ref: '#/components/parameters/Bucket'
get:
tags: [Multipart uploads]
operationId: listMultipartUploads
summary: List multipart uploads
description: Always returns an empty list; active multipart uploads cannot be enumerated.
responses:
'200':
description: Empty multipart upload listing.
content:
application/xml:
schema: { $ref: '#/components/schemas/ListMultipartUploadsResult' }
'403': { $ref: '#/components/responses/AccessDenied' }
/{bucket}/{key}:
parameters:
- $ref: '#/components/parameters/Bucket'
- $ref: '#/components/parameters/Key'
put:
tags: [Objects]
operationId: putObject
summary: Create or replace an object
parameters:
- $ref: '#/components/parameters/ContentType'
requestBody:
required: true
content:
application/octet-stream:
schema: { type: string, format: binary }
'*/*':
schema: { type: string, format: binary }
responses:
'200':
description: Object uploaded.
headers:
ETag: { $ref: '#/components/headers/ETag' }
'403': { $ref: '#/components/responses/AccessDenied' }
'501': { $ref: '#/components/responses/NotImplemented' }
get:
tags: [Objects]
operationId: getObject
summary: Download an object
parameters:
- name: Range
in: header
schema: { type: string, example: bytes=0-1023 }
responses:
'200':
description: Full object data.
headers:
ETag: { $ref: '#/components/headers/ETag' }
Content-Length: { $ref: '#/components/headers/ContentLength' }
Last-Modified: { $ref: '#/components/headers/LastModified' }
Accept-Ranges: { schema: { type: string, example: bytes } }
content:
application/octet-stream:
schema: { type: string, format: binary }
'206':
description: Requested byte range.
headers:
Content-Range: { schema: { type: string, example: bytes 0-1023/4096 } }
Content-Length: { $ref: '#/components/headers/ContentLength' }
content:
application/octet-stream:
schema: { type: string, format: binary }
'403': { $ref: '#/components/responses/AccessDenied' }
'404': { $ref: '#/components/responses/NoSuchKey' }
head:
tags: [Objects]
operationId: headObject
summary: Get object metadata
responses:
'200':
description: Object metadata in headers.
headers:
ETag: { $ref: '#/components/headers/ETag' }
Content-Length: { $ref: '#/components/headers/ContentLength' }
Last-Modified: { $ref: '#/components/headers/LastModified' }
Accept-Ranges: { schema: { type: string, example: bytes } }
'403': { $ref: '#/components/responses/AccessDenied' }
'404': { $ref: '#/components/responses/NoSuchKey' }
delete:
tags: [Objects]
operationId: deleteObject
summary: Delete an object
responses:
'204': { description: Object deleted. }
'403': { $ref: '#/components/responses/AccessDenied' }
'404': { $ref: '#/components/responses/NoSuchKey' }
/{bucket}/{key}?uploads:
parameters:
- $ref: '#/components/parameters/Bucket'
- $ref: '#/components/parameters/Key'
post:
tags: [Multipart uploads]
operationId: createMultipartUpload
summary: Initiate a multipart upload
parameters:
- $ref: '#/components/parameters/ContentType'
responses:
'200':
description: Upload initiated.
content:
application/xml:
schema: { $ref: '#/components/schemas/InitiateMultipartUploadResult' }
'403': { $ref: '#/components/responses/AccessDenied' }
'501': { $ref: '#/components/responses/NotImplemented' }
/{bucket}/{key}?uploadId={uploadId}&partNumber={partNumber}:
parameters:
- $ref: '#/components/parameters/Bucket'
- $ref: '#/components/parameters/Key'
- $ref: '#/components/parameters/UploadId'
- $ref: '#/components/parameters/PartNumber'
put:
tags: [Multipart uploads]
operationId: uploadPart
summary: Upload one multipart part
description: Parts must arrive strictly sequentially beginning with part number 1.
requestBody:
required: true
content:
application/octet-stream:
schema: { type: string, format: binary }
responses:
'200':
description: Part accepted.
headers:
ETag: { $ref: '#/components/headers/ETag' }
'400': { $ref: '#/components/responses/InvalidArgument' }
'403': { $ref: '#/components/responses/AccessDenied' }
'404': { $ref: '#/components/responses/NoSuchUpload' }
'503': { $ref: '#/components/responses/SlowDown' }
/{bucket}/{key}?uploadId={uploadId}:
parameters:
- $ref: '#/components/parameters/Bucket'
- $ref: '#/components/parameters/Key'
- $ref: '#/components/parameters/UploadId'
post:
tags: [Multipart uploads]
operationId: completeMultipartUpload
summary: Complete a multipart upload
requestBody:
required: true
content:
application/xml:
schema: { $ref: '#/components/schemas/CompleteMultipartUpload' }
responses:
'200':
description: Upload completed.
content:
application/xml:
schema: { $ref: '#/components/schemas/CompleteMultipartUploadResult' }
'400': { $ref: '#/components/responses/InvalidArgument' }
'403': { $ref: '#/components/responses/AccessDenied' }
'404': { $ref: '#/components/responses/NoSuchUpload' }
get:
tags: [Multipart uploads]
operationId: listParts
summary: List uploaded parts
parameters:
- name: part-number-marker
in: query
schema: { type: integer, minimum: 0, default: 0 }
- name: max-parts
in: query
schema: { type: integer, minimum: 1, maximum: 1000, default: 1000 }
responses:
'200':
description: Current part list.
content:
application/xml:
schema: { $ref: '#/components/schemas/ListPartsResult' }
'400': { $ref: '#/components/responses/InvalidArgument' }
'403': { $ref: '#/components/responses/AccessDenied' }
'404': { $ref: '#/components/responses/NoSuchUpload' }
delete:
tags: [Multipart uploads]
operationId: abortMultipartUpload
summary: Abort a multipart upload
responses:
'204': { description: Upload aborted. }
'403': { $ref: '#/components/responses/AccessDenied' }
'404': { $ref: '#/components/responses/NoSuchUpload' }
components:
securitySchemes:
sigv4:
type: apiKey
in: header
name: Authorization
description: AWS Signature Version 4 header authentication or equivalent `X-Amz-*` presigned query parameters.
parameters:
Bucket:
name: bucket
in: path
required: true
schema: { type: string }
Key:
name: key
in: path
required: true
schema: { type: string }
Prefix:
name: prefix
in: query
schema: { type: string, default: '' }
Delimiter:
name: delimiter
in: query
schema: { type: string, example: / }
UploadId:
name: uploadId
in: query
required: true
schema: { type: string }
PartNumber:
name: partNumber
in: query
required: true
schema: { type: integer, minimum: 1, maximum: 10000 }
ContentType:
name: Content-Type
in: header
schema: { type: string, example: application/octet-stream }
headers:
ETag:
schema: { type: string, example: '"d41d8cd98f00b204e9800998ecf8427e"' }
ContentLength:
schema: { type: integer, minimum: 0 }
LastModified:
schema: { type: string, format: date-time }
responses:
AccessDenied:
description: Authentication failed, the access key is not recognized, the presigned URL expired, or bucket access is denied.
content: { application/xml: { schema: { $ref: '#/components/schemas/S3Error' } } }
NoSuchKey:
description: The object does not exist.
content: { application/xml: { schema: { $ref: '#/components/schemas/S3Error' } } }
NoSuchUpload:
description: The multipart upload does not exist.
content: { application/xml: { schema: { $ref: '#/components/schemas/S3Error' } } }
InvalidArgument:
description: A required parameter or body is invalid.
content: { application/xml: { schema: { $ref: '#/components/schemas/S3Error' } } }
NotImplemented:
description: This S3 operation is not supported.
content: { application/xml: { schema: { $ref: '#/components/schemas/S3Error' } } }
SlowDown:
description: Parts are out of sequence or the upload is busy; retry later.
content: { application/xml: { schema: { $ref: '#/components/schemas/S3Error' } } }
MethodNotAllowed:
description: The method/path combination is unsupported.
content: { application/xml: { schema: { $ref: '#/components/schemas/S3Error' } } }
schemas:
S3Error:
type: object
xml: { name: Error }
required: [Code, Message, RequestId]
properties:
Code: { type: string, xml: { name: Code } }
Message: { type: string, xml: { name: Message } }
Resource: { type: string, xml: { name: Resource } }
RequestId: { type: string, xml: { name: RequestId } }
Object:
type: object
xml: { name: Contents }
properties:
Key: { type: string }
LastModified: { type: string, format: date-time }
ETag: { type: string }
Size: { type: integer }
StorageClass: { type: string, example: STANDARD }
ListBucketResult:
type: object
xml: { name: ListBucketResult, namespace: 'http://s3.amazonaws.com/doc/2006-03-01/' }
properties:
Name: { type: string }
Prefix: { type: string }
Delimiter: { type: string }
MaxKeys: { type: integer, example: 1000 }
IsTruncated: { type: boolean }
Contents: { type: array, items: { $ref: '#/components/schemas/Object' }, xml: { wrapped: false } }
InitiateMultipartUploadResult:
type: object
xml: { name: InitiateMultipartUploadResult, namespace: 'http://s3.amazonaws.com/doc/2006-03-01/' }
required: [Bucket, Key, UploadId]
properties:
Bucket: { type: string }
Key: { type: string }
UploadId: { type: string }
CompleteMultipartUpload:
type: object
xml: { name: CompleteMultipartUpload }
required: [Part]
properties:
Part:
type: array
minItems: 1
xml: { wrapped: false }
items:
type: object
xml: { name: Part }
required: [PartNumber, ETag]
properties:
PartNumber: { type: integer }
ETag: { type: string }
CompleteMultipartUploadResult:
type: object
xml: { name: CompleteMultipartUploadResult, namespace: 'http://s3.amazonaws.com/doc/2006-03-01/' }
properties:
Location: { type: string }
Bucket: { type: string }
Key: { type: string }
ETag: { type: string }
ListPartsResult:
type: object
xml: { name: ListPartsResult, namespace: 'http://s3.amazonaws.com/doc/2006-03-01/' }
properties:
Bucket: { type: string }
Key: { type: string }
UploadId: { type: string }
NextPartNumberMarker: { type: integer }
IsTruncated: { type: boolean }
Part:
type: array
xml: { wrapped: false }
items:
type: object
properties:
PartNumber: { type: integer }
LastModified: { type: string, format: date-time }
ETag: { type: string }
Size: { type: integer }
ListMultipartUploadsResult:
type: object
xml: { name: ListMultipartUploadsResult, namespace: 'http://s3.amazonaws.com/doc/2006-03-01/' }
properties:
Bucket: { type: string }
KeyMarker: { type: string }
UploadIdMarker: { type: string }
NextKeyMarker: { type: string }
NextUploadIdMarker: { type: string }
MaxUploads: { type: integer, example: 1000 }
IsTruncated: { type: boolean, example: false }
+2 -5
View File
@@ -20,9 +20,9 @@
// Optional: --port 8976 (must not be in use; Desktop-app clients accept any
// loopback port automatically, no need to register it in Google Cloud Console).
import { exec } from "node:child_process";
import { randomBytes } from "node:crypto";
import http from "node:http";
import { exec } from "node:child_process";
import { URL } from "node:url";
const SCOPE = "https://www.googleapis.com/auth/drive";
@@ -137,10 +137,7 @@ async function main() {
process.exit(1);
}
if (!tokenData.refresh_token) {
console.error(
"No refresh_token was returned. This usually means the account already has an active grant.\n" +
"Revoke it at https://myaccount.google.com/permissions and re-run this script.",
);
console.error("No refresh_token was returned. This usually means the account already has an active grant.\n" + "Revoke it at https://myaccount.google.com/permissions and re-run this script.");
process.exit(1);
}
+56 -6
View File
@@ -1,5 +1,11 @@
import type { S3ErrorCode } from "./s3-errors";
import type { Env } from "./types";
const MAX_PRESIGN_EXPIRY_SECONDS = 7 * 24 * 60 * 60;
const MAX_CLOCK_SKEW_MS = 15 * 60 * 1000;
export type VerifyResult = { ok: true } | { ok: false; code: S3ErrorCode; message?: string };
function encodeRFC3986(str: string): string {
return encodeURIComponent(str).replace(/[!'()*]/g, (c) => `%${c.charCodeAt(0).toString(16).toUpperCase()}`);
}
@@ -84,8 +90,35 @@ async function createCanonicalRequest(request: Request, isQueryAuth: boolean): P
return [method, canonicalUri, params, canonicalHeaders, signedHeaders, payloadHash].join("\n");
}
function parseAmzDate(datetime: string): number | null {
const match = /^(\d{4})(\d{2})(\d{2})T(\d{2})(\d{2})(\d{2})Z$/.exec(datetime);
if (!match) return null;
const [, year, month, day, hour, minute, second] = match;
const timestamp = Date.UTC(Number(year), Number(month) - 1, Number(day), Number(hour), Number(minute), Number(second));
const parsed = new Date(timestamp);
if (parsed.getUTCFullYear() !== Number(year) || parsed.getUTCMonth() !== Number(month) - 1 || parsed.getUTCDate() !== Number(day) || parsed.getUTCHours() !== Number(hour) || parsed.getUTCMinutes() !== Number(minute) || parsed.getUTCSeconds() !== Number(second)) {
return null;
}
return timestamp;
}
function constantTimeEqual(left: string, right: string): boolean {
if (left.length !== right.length) return false;
let mismatch = 0;
for (let index = 0; index < left.length; index++) {
mismatch |= left.charCodeAt(index) ^ right.charCodeAt(index);
}
return mismatch === 0;
}
function signatureMismatch(): VerifyResult {
return { ok: false, code: "SignatureDoesNotMatch" };
}
/** Verifies an AWS Signature V4 signature carried in either the Authorization header or presigned query params. */
export async function verifySignature(request: Request, env: Env): Promise<boolean> {
export async function verifySignature(request: Request, env: Env): Promise<VerifyResult> {
const url = new URL(request.url);
const headers = request.headers;
@@ -99,15 +132,32 @@ export async function verifySignature(request: Request, env: Env): Promise<boole
algorithm = authHeader.split(" ")[0];
}
if (!algorithm?.includes("AWS4-HMAC-SHA256")) {
return false;
if (algorithm !== "AWS4-HMAC-SHA256") {
return signatureMismatch();
}
const datetime = (isQueryAuth ? url.searchParams.get("X-Amz-Date") : headers.get("x-amz-date")) ?? "";
if (!datetime) return isQueryAuth ? { ok: false, code: "AccessDenied", message: "Request has expired" } : signatureMismatch();
const signedAt = parseAmzDate(datetime);
if (signedAt === null) return isQueryAuth ? { ok: false, code: "AccessDenied", message: "Request has expired" } : signatureMismatch();
if (!datetime) return false;
if (isQueryAuth) {
const expires = url.searchParams.get("X-Amz-Expires");
if (!expires || !/^\d+$/.test(expires)) return { ok: false, code: "AccessDenied", message: "Request has expired" };
const expiresIn = Number(expires);
if (!Number.isSafeInteger(expiresIn) || expiresIn < 1 || expiresIn > MAX_PRESIGN_EXPIRY_SECONDS || Date.now() > signedAt + expiresIn * 1000) {
return { ok: false, code: "AccessDenied", message: "Request has expired" };
}
} else if (Math.abs(Date.now() - signedAt) > MAX_CLOCK_SKEW_MS) {
return { ok: false, code: "RequestTimeTooSkewed" };
}
const date = datetime.substring(0, 8);
const credential = isQueryAuth ? (url.searchParams.get("X-Amz-Credential") ?? "") : (/Credential=([^,\s]+)/.exec(headers.get("Authorization") ?? "")?.[1] ?? "");
const credentialParts = credential.split("/");
if (credentialParts.length !== 5 || credentialParts[0] !== env.ACCESS_KEY || credentialParts.slice(1).join("/") !== `${date}/${env.REGION}/s3/aws4_request`) {
return { ok: false, code: "AccessDenied" };
}
const canonicalRequest = await createCanonicalRequest(request, isQueryAuth);
const hashedCanonicalRequest = await sha256(canonicalRequest);
@@ -124,9 +174,9 @@ export async function verifySignature(request: Request, env: Env): Promise<boole
expectedSignature = url.searchParams.get("X-Amz-Signature") ?? "";
} else {
const authHeader = headers.get("Authorization") ?? "";
const match = authHeader.match(/Signature=([a-f0-9]+)/);
const match = authHeader.match(/Signature=([a-fA-F0-9]+)/);
expectedSignature = match ? match[1] : "";
}
return signatureHex === expectedSignature;
return constantTimeEqual(signatureHex, expectedSignature) ? { ok: true } : signatureMismatch();
}
+69
View File
@@ -0,0 +1,69 @@
import type { Env } from "./types";
const ALLOWED_METHODS = "GET, HEAD, PUT, POST, DELETE, OPTIONS";
const DEFAULT_ALLOWED_HEADERS = "Authorization, Content-Type, Content-Length, Content-MD5, Content-Encoding, Range, x-amz-content-sha256, x-amz-copy-source, x-amz-date, x-amz-decoded-content-length, x-amz-mp-object-size, x-amz-security-token";
const EXPOSED_HEADERS = "ETag, Content-Range, Content-Length, Last-Modified, Accept-Ranges, x-amz-request-id";
function configuredOrigins(env: Env): string[] {
return (env.CORS_ALLOWED_ORIGINS ?? "")
.split(",")
.map((origin) => origin.trim())
.filter(Boolean);
}
function hasWildcardOrigin(env: Env): boolean {
return env.CORS_ALLOWED_ORIGINS?.trim() === "*";
}
function appendVary(headers: Headers, value: string): void {
const existing = headers.get("Vary");
if (!existing) {
headers.set("Vary", value);
return;
}
if (!existing.split(",").some((part) => part.trim().toLowerCase() === value.toLowerCase())) {
headers.set("Vary", `${existing}, ${value}`);
}
}
/** Returns the permitted value for Access-Control-Allow-Origin, or null when CORS is disabled or denied. */
export function resolveOrigin(request: Request, env: Env): string | null {
const origin = request.headers.get("Origin");
if (!origin) return null;
if (hasWildcardOrigin(env)) return "*";
return configuredOrigins(env).includes(origin) ? origin : null;
}
/** Builds an unauthenticated browser preflight response. */
export function preflightResponse(request: Request, env: Env): Response {
const headers = new Headers();
const origin = resolveOrigin(request, env);
if (origin) {
headers.set("Access-Control-Allow-Origin", origin);
headers.set("Access-Control-Allow-Methods", ALLOWED_METHODS);
headers.set("Access-Control-Allow-Headers", request.headers.get("Access-Control-Request-Headers") ?? DEFAULT_ALLOWED_HEADERS);
headers.set("Access-Control-Max-Age", "86400");
if (!hasWildcardOrigin(env)) appendVary(headers, "Origin");
}
return new Response(null, { status: 204, headers });
}
/** Adds CORS response headers without mutating the response supplied by the request handler. */
export function withCors(response: Response, request: Request, env: Env): Response {
const headers = new Headers(response.headers);
if (env.CORS_ALLOWED_ORIGINS?.trim() && !hasWildcardOrigin(env)) {
appendVary(headers, "Origin");
}
const origin = resolveOrigin(request, env);
if (origin) {
headers.set("Access-Control-Allow-Origin", origin);
headers.set("Access-Control-Expose-Headers", EXPOSED_HEADERS);
}
return new Response(response.body, { status: response.status, statusText: response.statusText, headers });
}
+36
View File
@@ -0,0 +1,36 @@
import openApiSpec from "../docs/openapi.yaml";
export const OPENAPI_PATH = "/openapi.yaml";
export const DOCS_PATH = "/docs";
export function openApiResponse(): Response {
return new Response(openApiSpec, {
headers: {
"Content-Type": "application/yaml; charset=utf-8",
"Cache-Control": "public, max-age=300",
},
});
}
export function docsResponse(): Response {
const html = `<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="color-scheme" content="dark light">
<title>I.R.I.S. S3 API reference</title>
</head>
<body>
<script id="api-reference" data-url="${OPENAPI_PATH}"></script>
<script src="https://cdn.jsdelivr.net/npm/@scalar/api-reference"></script>
</body>
</html>`;
return new Response(html, {
headers: {
"Content-Type": "text/html; charset=utf-8",
"Cache-Control": "public, max-age=300",
},
});
}
+2 -1
View File
@@ -220,9 +220,10 @@ export async function streamDownloadFromDrive(accessToken: string, bucket: strin
console.error(await downloadRes.text());
throw new Error("Download failed");
}
if (!downloadRes.body) throw new Error("Download response had no body");
return {
body: downloadRes.body!,
body: downloadRes.body,
contentType: file.mimeType || "application/octet-stream",
size: parseInt(file.size || "0", 10),
id: file.id,
+15 -7
View File
@@ -1,5 +1,7 @@
import { verifySignature } from "./aws-signature";
import { isAllowedBucket, isPublicReadBucket } from "./bucket-access";
import { preflightResponse, withCors } from "./cors";
import * as docs from "./docs";
import { getAccessToken } from "./google-drive";
import { MultipartUploadDO } from "./multipart-do";
import { dispatch } from "./router";
@@ -10,27 +12,33 @@ export { MultipartUploadDO };
export default {
async fetch(request: Request, env: Env, _ctx: ExecutionContext): Promise<Response> {
if (request.method === "OPTIONS") return new Response(null, { status: 204 });
if (request.method === "OPTIONS") return preflightResponse(request, env);
const url = new URL(request.url);
if (env.ENABLE_DOCS !== "false") {
if (request.method === "GET" && url.pathname === docs.OPENAPI_PATH) return withCors(docs.openApiResponse(), request, env);
if (request.method === "GET" && url.pathname === docs.DOCS_PATH && !isAllowedBucket("docs", env)) return withCors(docs.docsResponse(), request, env);
}
const pathParts = url.pathname.split("/").filter(Boolean);
const bucket = pathParts[0] || "";
const objectKey = pathParts.slice(1).join("/");
const resource = url.pathname || "/";
try {
if (!isAllowedBucket(bucket, env)) return s3Error("AccessDenied", 403, undefined, resource, request.method === "HEAD");
if (!isAllowedBucket(bucket, env)) return withCors(s3Error("AccessDenied", 403, undefined, resource, request.method === "HEAD"), request, env);
const isPublicRead = isPublicReadBucket(bucket, env) && (request.method === "GET" || request.method === "HEAD");
if (!isPublicRead && !(await verifySignature(request, env))) {
return s3Error("SignatureDoesNotMatch", 403, undefined, resource, request.method === "HEAD");
const signature = isPublicRead ? { ok: true as const } : await verifySignature(request, env);
if (!signature.ok) {
return withCors(s3Error(signature.code, 403, signature.message, resource, request.method === "HEAD"), request, env);
}
return await dispatch(request, env, await getAccessToken(env), bucket, objectKey);
return withCors(await dispatch(request, env, await getAccessToken(env), bucket, objectKey), request, env);
} catch (error) {
if (error instanceof S3Exception) return s3Error(error.code, error.status, error.message, resource, request.method === "HEAD", error.headers);
if (error instanceof S3Exception) return withCors(s3Error(error.code, error.status, error.message, resource, request.method === "HEAD", error.headers), request, env);
console.error(JSON.stringify({ message: "request failed", error: error instanceof Error ? error.message : String(error), method: request.method, path: url.pathname }));
return s3Error("InternalError", 500, undefined, resource, request.method === "HEAD");
return withCors(s3Error("InternalError", 500, undefined, resource, request.method === "HEAD"), request, env);
}
},
} satisfies ExportedHandler<Env>;
+14 -8
View File
@@ -57,6 +57,12 @@ export class MultipartUploadDO extends DurableObject<Env> {
return this.ctx.storage.sql.exec<StateValueRow>("SELECT v FROM state WHERE k = ?", key).toArray()[0]?.v as T | undefined;
}
private requiredValue<T extends ArrayBuffer | string | number>(key: string): T {
const value = this.getValue<T>(key);
if (value === undefined) throw new Error(`Missing multipart upload state: ${key}`);
return value;
}
private setValue(key: string, value: ArrayBuffer | string | number | null): void {
this.ctx.storage.sql.exec("INSERT INTO state (k, v) VALUES (?, ?) ON CONFLICT(k) DO UPDATE SET v = excluded.v", key, value);
}
@@ -97,7 +103,7 @@ export class MultipartUploadDO extends DurableObject<Env> {
private async resyncExpiredLease(inFlight: InFlight, requestId: string, partLen: number): Promise<BeginPartResult | null> {
if (partLen !== inFlight.partLen) return { kind: "error", code: "InvalidPart", message: "A retried part must have the same decoded length" };
const accessToken = await getAccessToken(this.env);
const actualOffset = await queryStatus(this.getValue<string>("uploadUrl")!, accessToken);
const actualOffset = await queryStatus(this.requiredValue<string>("uploadUrl"), accessToken);
const partStartFileOffset = inFlight.driveOffsetAtStart + inFlight.carryLen;
let skipBytes = 0;
if (actualOffset === inFlight.driveOffsetAtStart) {
@@ -129,7 +135,7 @@ export class MultipartUploadDO extends DurableObject<Env> {
leaseExpiresAt: Date.now() + LEASE_MS,
};
this.setValue("inFlight", JSON.stringify(inFlight));
return { kind: "admit", uploadUrl: this.getValue<string>("uploadUrl")!, driveOffset, carry, sendLen, skipBytes };
return { kind: "admit", uploadUrl: this.requiredValue<string>("uploadUrl"), driveOffset, carry, sendLen, skipBytes };
}
async beginPart(requestId: string, partNumber: number, partLen: number): Promise<BeginPartResult> {
@@ -229,15 +235,15 @@ export class MultipartUploadDO extends DurableObject<Env> {
const accessToken = await getAccessToken(this.env);
let metadata: DriveUploadResult;
if (total === 0) {
await cancelSession(this.getValue<string>("uploadUrl")!, accessToken);
await cancelSession(this.requiredValue<string>("uploadUrl"), accessToken);
metadata = await createEmptyFile(accessToken, {
name: this.getValue<string>("fileName")!,
parents: [this.getValue<string>("parentFolderId")!],
mimeType: this.getValue<string>("mimeType")!,
name: this.requiredValue<string>("fileName"),
parents: [this.requiredValue<string>("parentFolderId")],
mimeType: this.requiredValue<string>("mimeType"),
existingFileId: this.getValue<string>("existingFileId"),
});
} else {
metadata = await putFinalChunk(this.getValue<string>("uploadUrl")!, accessToken, driveOffset, total, carry);
metadata = await putFinalChunk(this.requiredValue<string>("uploadUrl"), accessToken, driveOffset, total, carry);
}
const partEtags = stored.map((part) => part.etag);
await this.ctx.storage.deleteAlarm();
@@ -259,7 +265,7 @@ export class MultipartUploadDO extends DurableObject<Env> {
async abort(): Promise<boolean> {
if (!this.hasUpload()) return false;
const accessToken = await getAccessToken(this.env);
await cancelSession(this.getValue<string>("uploadUrl")!, accessToken);
await cancelSession(this.requiredValue<string>("uploadUrl"), accessToken);
await this.ctx.storage.deleteAlarm();
await this.ctx.storage.deleteAll();
for (const waiter of this.waiters.values()) {
+2 -1
View File
@@ -1,10 +1,11 @@
import { escapeXml } from "./s3-xml";
export type S3ErrorCode = "AccessDenied" | "SignatureDoesNotMatch" | "NoSuchKey" | "NoSuchUpload" | "InvalidPart" | "InvalidPartOrder" | "EntityTooLarge" | "MalformedXML" | "InvalidArgument" | "MethodNotAllowed" | "NotImplemented" | "SlowDown" | "InternalError";
export type S3ErrorCode = "AccessDenied" | "SignatureDoesNotMatch" | "RequestTimeTooSkewed" | "NoSuchKey" | "NoSuchUpload" | "InvalidPart" | "InvalidPartOrder" | "EntityTooLarge" | "MalformedXML" | "InvalidArgument" | "MethodNotAllowed" | "NotImplemented" | "SlowDown" | "InternalError";
const DEFAULT_MESSAGES: Record<S3ErrorCode, string> = {
AccessDenied: "Access Denied",
SignatureDoesNotMatch: "The request signature we calculated does not match the signature you provided.",
RequestTimeTooSkewed: "The difference between the request time and the server's time is too large.",
NoSuchKey: "The specified key does not exist.",
NoSuchUpload: "The specified multipart upload does not exist.",
InvalidPart: "One or more of the specified parts could not be found.",
+4
View File
@@ -0,0 +1,4 @@
declare module "*.yaml" {
const content: string;
export default content;
}
+2
View File
@@ -12,6 +12,8 @@ export interface Env {
PUBLIC_READ_BUCKETS?: string;
ALLOW_MULTIPART?: string;
ETAG_STYLE?: "md5" | "multipart";
CORS_ALLOWED_ORIGINS?: string;
ENABLE_DOCS?: string;
}
export interface GoogleDriveFile {
+119
View File
@@ -0,0 +1,119 @@
import { AwsClient } from "aws4fetch";
import { afterEach, describe, expect, it, vi } from "vitest";
import { preflightResponse, withCors } from "../src/cors";
import worker from "../src/index";
import type { Env } from "../src/types";
import { env } from "cloudflare:test";
const ENV = env as unknown as Env;
const ENDPOINT = "https://s3-api.example.com";
const ORIGIN = "http://localhost:5173";
const CTX = { waitUntil: vi.fn(), passThroughOnException: vi.fn() } as unknown as ExecutionContext;
async function signed(path: string, init: RequestInit): Promise<Request> {
const aws = new AwsClient({ accessKeyId: ENV.ACCESS_KEY, secretAccessKey: ENV.SECRET_KEY, region: ENV.REGION, service: "s3" });
const bodyLength = typeof init.body === "string" ? new TextEncoder().encode(init.body).byteLength : init.body instanceof Uint8Array ? init.body.byteLength : undefined;
return aws.sign(`${ENDPOINT}${path}`, {
...init,
headers: { "x-amz-content-sha256": "UNSIGNED-PAYLOAD", ...(bodyLength === undefined ? {} : { "x-amz-decoded-content-length": String(bodyLength) }), ...init.headers },
});
}
function fakeGoogleFetch(input: string | URL | Request, init?: RequestInit): Promise<Response> {
const request = input instanceof Request ? input : new Request(input, init);
const url = new URL(request.url);
if (url.hostname === "oauth2.googleapis.com") return Promise.resolve(Response.json({ access_token: "token", expires_in: 3600 }));
if (url.pathname === "/drive/v3/files" && request.method === "GET") return Promise.resolve(Response.json({ files: [] }));
if (url.pathname === "/drive/v3/files" && request.method === "POST") return Promise.resolve(Response.json({ id: "folder-1" }));
if (url.pathname.startsWith("/upload/drive/v3/files")) return Promise.resolve(new Response(null, { headers: { Location: "https://www.googleapis.com/upload/session/test" } }));
if (url.pathname === "/upload/session/test") return Promise.resolve(Response.json({ id: "file-1", name: "file.txt", md5Checksum: "d41d8cd98f00b204e9800998ecf8427e" }));
return Promise.resolve(new Response("Not Found", { status: 404 }));
}
afterEach(() => vi.unstubAllGlobals());
describe("CORS", () => {
it("returns allow-list preflight headers", async () => {
const response = preflightResponse(
new Request(`${ENDPOINT}/test-bucket/file.txt`, {
method: "OPTIONS",
headers: { Origin: ORIGIN, "Access-Control-Request-Method": "PUT", "Access-Control-Request-Headers": "content-type,x-amz-date" },
}),
ENV,
);
expect(response.status).toBe(204);
expect(response.headers.get("Access-Control-Allow-Origin")).toBe(ORIGIN);
expect(response.headers.get("Access-Control-Allow-Methods")).toBe("GET, HEAD, PUT, POST, DELETE, OPTIONS");
expect(response.headers.get("Access-Control-Allow-Headers")).toBe("content-type,x-amz-date");
expect(response.headers.get("Access-Control-Max-Age")).toBe("86400");
expect(response.headers.get("Vary")).toBe("Origin");
});
it("returns a bare 204 for a disallowed origin", async () => {
const response = preflightResponse(new Request(`${ENDPOINT}/test-bucket/file.txt`, { method: "OPTIONS", headers: { Origin: "https://not-allowed.example", "Access-Control-Request-Method": "PUT" } }), ENV);
expect(response.status).toBe(204);
expect([...response.headers]).toEqual([]);
});
it("keeps OPTIONS requests without an Origin byte-compatible with the prior bare 204", () => {
const response = preflightResponse(new Request(`${ENDPOINT}/test-bucket/file.txt`, { method: "OPTIONS" }), ENV);
expect(response.status).toBe(204);
expect([...response.headers]).toEqual([]);
});
it("exposes ETag for successful PUT responses", async () => {
vi.stubGlobal("fetch", vi.fn(fakeGoogleFetch));
const request = await signed("/test-bucket/file.txt", { method: "PUT", body: "hello", headers: { Origin: ORIGIN, "Content-Type": "text/plain" } });
const response = await worker.fetch(request, ENV, CTX);
expect(response.status).toBe(200);
expect(response.headers.get("ETag")).toBe('"d41d8cd98f00b204e9800998ecf8427e"');
expect(response.headers.get("Access-Control-Allow-Origin")).toBe(ORIGIN);
expect(response.headers.get("Access-Control-Expose-Headers")).toContain("ETag");
});
it("adds CORS headers to access-denied and missing-key errors", async () => {
const denied = await worker.fetch(new Request(`${ENDPOINT}/not-a-bucket`, { headers: { Origin: ORIGIN } }), ENV, CTX);
expect(denied.status).toBe(403);
expect(denied.headers.get("Access-Control-Allow-Origin")).toBe(ORIGIN);
vi.stubGlobal("fetch", vi.fn(fakeGoogleFetch));
const missing = await worker.fetch(await signed("/test-bucket/missing.txt", { method: "GET", headers: { Origin: ORIGIN } }), ENV, CTX);
expect(missing.status).toBe(404);
expect(await missing.text()).toContain("<Code>NoSuchKey</Code>");
expect(missing.headers.get("Access-Control-Allow-Origin")).toBe(ORIGIN);
});
it("does not alter the S3 response for a disallowed Origin", async () => {
const response = await worker.fetch(new Request(`${ENDPOINT}/not-a-bucket`, { headers: { Origin: "https://not-allowed.example" } }), ENV, CTX);
expect(response.status).toBe(403);
expect(await response.text()).toContain("<Code>AccessDenied</Code>");
expect(response.headers.get("Access-Control-Allow-Origin")).toBeNull();
expect(response.headers.get("Access-Control-Expose-Headers")).toBeNull();
expect(response.headers.get("Vary")).toBe("Origin");
});
it("does not attach Access-Control headers to server-to-server responses without Origin", () => {
const response = withCors(new Response("S3 response", { status: 200, headers: { ETag: '"etag"' } }), new Request(`${ENDPOINT}/test-bucket/file.txt`), ENV);
expect(response.status).toBe(200);
expect(response.headers.get("ETag")).toBe('"etag"');
expect(response.headers.get("Access-Control-Allow-Origin")).toBeNull();
expect(response.headers.get("Access-Control-Expose-Headers")).toBeNull();
expect(response.headers.get("Vary")).toBe("Origin");
});
it("emits no CORS headers when CORS_ALLOWED_ORIGINS is unset", () => {
const request = new Request(`${ENDPOINT}/test-bucket/file.txt`, { headers: { Origin: ORIGIN } });
const response = withCors(new Response(null, { headers: { ETag: '"etag"' } }), request, { ...ENV, CORS_ALLOWED_ORIGINS: undefined });
expect(response.headers.get("Access-Control-Allow-Origin")).toBeNull();
expect(response.headers.get("Access-Control-Expose-Headers")).toBeNull();
expect(response.headers.get("Vary")).toBeNull();
});
});
+39
View File
@@ -0,0 +1,39 @@
import { describe, expect, it, vi } from "vitest";
import worker from "../src/index";
import type { Env } from "../src/types";
import { env } from "cloudflare:test";
const ENV = env as unknown as Env;
const ENDPOINT = "https://s3-api.example.com";
const CTX = { waitUntil: vi.fn(), passThroughOnException: vi.fn() } as unknown as ExecutionContext;
describe("API documentation routes", () => {
it("serves the Scalar shell and raw OpenAPI document", async () => {
const docs = await worker.fetch(new Request(`${ENDPOINT}/docs`), ENV, CTX);
expect(docs.status).toBe(200);
expect(docs.headers.get("Content-Type")).toContain("text/html");
expect(await docs.text()).toContain("@scalar/api-reference");
const spec = await worker.fetch(new Request(`${ENDPOINT}/openapi.yaml`), ENV, CTX);
expect(spec.status).toBe(200);
expect(spec.headers.get("Content-Type")).toContain("application/yaml");
expect(new TextDecoder().decode(await spec.arrayBuffer())).toContain("openapi: 3.1.0");
});
it("bypasses documentation routes when disabled", async () => {
const disabled = { ...ENV, ENABLE_DOCS: "false" };
for (const path of ["/docs", "/openapi.yaml"]) {
const response = await worker.fetch(new Request(`${ENDPOINT}${path}`), disabled, CTX);
expect(response.status).toBe(403);
}
});
it("does not claim /docs when docs is a configured bucket", async () => {
const withDocsBucket = { ...ENV, ALLOWED_BUCKETS: "test-bucket,docs" };
const response = await worker.fetch(new Request(`${ENDPOINT}/docs`), withDocsBucket, CTX);
expect(response.status).toBe(403);
expect(await response.text()).toContain("<Code>SignatureDoesNotMatch</Code>");
});
});
+64 -9
View File
@@ -56,7 +56,9 @@ class FakeDrive {
if (url.pathname === "/drive/v3/files" && request.method === "POST") return this.createMetadata(await request.json<Record<string, unknown>>());
if (url.pathname.startsWith("/drive/v3/files/") && url.searchParams.get("alt") === "media") return this.download(url, request);
if (url.pathname.startsWith("/drive/v3/files/") && request.method === "DELETE") {
this.files.delete(url.pathname.split("/").at(-1)!);
const fileId = url.pathname.split("/").at(-1);
if (!fileId) return new Response("Not Found", { status: 404 });
this.files.delete(fileId);
return new Response(null, { status: 204 });
}
if (url.pathname.startsWith("/upload/drive/v3/files") && url.searchParams.get("uploadType") === "resumable") return this.initialize(url, request);
@@ -121,7 +123,8 @@ class FakeDrive {
}
private async upload(url: URL, request: Request): Promise<Response> {
const id = url.pathname.split("/").at(-1)!;
const id = url.pathname.split("/").at(-1);
if (!id) return new Response(null, { status: 404 });
const session = this.sessions.get(id);
if (!session) return new Response(null, { status: 404 });
if (request.method === "DELETE") {
@@ -159,7 +162,9 @@ class FakeDrive {
}
private download(url: URL, request: Request): Response {
const file = this.files.get(url.pathname.split("/").at(-1)!);
const fileId = url.pathname.split("/").at(-1);
if (!fileId) return new Response(null, { status: 404 });
const file = this.files.get(fileId);
if (!file) return new Response(null, { status: 404 });
const range = request.headers.get("Range");
if (!range) return new Response(file.data, { headers: { "Content-Length": String(file.data.byteLength) } });
@@ -204,6 +209,18 @@ async function signed(path: string, init: RequestInit): Promise<Request> {
});
}
async function presigned(path: string, init: RequestInit, options: { datetime?: string; accessKeyId?: string; expires?: string } = {}): Promise<Request> {
const url = new URL(`${ENDPOINT}${path}`);
url.searchParams.set("X-Amz-Expires", options.expires ?? "60");
const aws = new AwsClient({ accessKeyId: options.accessKeyId ?? ENV.ACCESS_KEY, secretAccessKey: ENV.SECRET_KEY, region: ENV.REGION, service: "s3" });
const bodyLength = typeof init.body === "string" ? new TextEncoder().encode(init.body).byteLength : init.body instanceof Uint8Array ? init.body.byteLength : undefined;
return aws.sign(url.toString(), {
...init,
headers: { "x-amz-content-sha256": "UNSIGNED-PAYLOAD", ...(bodyLength === undefined ? {} : { "x-amz-decoded-content-length": String(bodyLength) }), ...init.headers },
aws: { signQuery: true, ...(options.datetime ? { datetime: options.datetime } : {}) },
});
}
let drive: FakeDrive;
beforeEach(async () => {
@@ -226,7 +243,10 @@ describe("S3 compatibility", () => {
const second = await worker.fetch(await signed("/test-bucket/file.txt", { method: "PUT", body: "second" }), ENV, CTX);
expect(second.status).toBe(200);
expect([...drive.files.values()].filter((file) => file.name === "file.txt")).toHaveLength(1);
expect(new TextDecoder().decode([...drive.files.values()].find((file) => file.name === "file.txt")!.data)).toBe("second");
const stored = [...drive.files.values()].find((file) => file.name === "file.txt");
expect(stored).toBeDefined();
if (!stored) throw new Error("Overwritten object was not stored");
expect(new TextDecoder().decode(stored.data)).toBe("second");
});
it("sets Last-Modified on GET and HEAD from Drive's modifiedTime", async () => {
@@ -264,6 +284,33 @@ describe("S3 compatibility", () => {
expect(response.status).toBe(200);
});
it("rejects expired presigned URLs and accepts unexpired ones", async () => {
const expired = await worker.fetch(await presigned("/test-bucket/file.txt", { method: "GET" }, { datetime: "20200101T000000Z", expires: "60" }), ENV, CTX);
expect(expired.status).toBe(403);
expect(await expired.text()).toContain("<Code>AccessDenied</Code>");
});
it("accepts an unexpired presigned URL", async () => {
await worker.fetch(await signed("/test-bucket/presigned.txt", { method: "PUT", body: "hello" }), ENV, CTX);
const response = await worker.fetch(await presigned("/test-bucket/presigned.txt", { method: "GET" }), ENV, CTX);
expect(response.status).toBe(200);
expect(await response.text()).toBe("hello");
});
it("rejects a Credential access key that does not match ACCESS_KEY", async () => {
const response = await worker.fetch(await presigned("/test-bucket/file.txt", { method: "GET" }, { accessKeyId: "unexpected-access-key" }), ENV, CTX);
expect(response.status).toBe(403);
expect(await response.text()).toContain("<Code>AccessDenied</Code>");
});
it("rejects header-authenticated requests outside the 15-minute clock skew", async () => {
const aws = new AwsClient({ accessKeyId: ENV.ACCESS_KEY, secretAccessKey: ENV.SECRET_KEY, region: ENV.REGION, service: "s3" });
const request = await aws.sign(`${ENDPOINT}/test-bucket/file.txt`, { method: "GET", aws: { datetime: "20200101T000000Z" } });
const response = await worker.fetch(request, ENV, CTX);
expect(response.status).toBe(403);
expect(await response.text()).toContain("<Code>RequestTimeTooSkewed</Code>");
});
it("decodes both aws-chunked framing variants across arbitrary boundaries", async () => {
const payload = bytes(70_013);
for (const trailer of [true, false]) {
@@ -282,20 +329,28 @@ describe("S3 compatibility", () => {
const source = bytes(1_500_123);
const create = await worker.fetch(await signed("/test-bucket/big.bin?uploads", { method: "POST", headers: { "Content-Type": "application/octet-stream" } }), ENV, CTX);
expect(create.status).toBe(200);
const uploadId = /<UploadId>([^<]+)<\/UploadId>/.exec(await create.text())![1];
const uploadId = /<UploadId>([^<]+)<\/UploadId>/.exec(await create.text())?.[1];
expect(uploadId).toBeDefined();
if (!uploadId) throw new Error("Multipart initiation did not return an upload ID");
const completed: Array<{ partNumber: number; etag: string }> = [];
for (let index = 0, offset = 0; offset < source.byteLength; index++) {
const end = Math.min(source.byteLength, offset + 500_000);
const part = await worker.fetch(await signed(`/test-bucket/big.bin?partNumber=${index + 1}&uploadId=${encodeURIComponent(uploadId)}`, { method: "PUT", body: source.slice(offset, end) }), ENV, CTX);
expect(part.status).toBe(200);
completed.push({ partNumber: index + 1, etag: part.headers.get("ETag")!.replaceAll('"', "") });
const partEtag = part.headers.get("ETag");
expect(partEtag).toBeDefined();
if (!partEtag) throw new Error(`Multipart part ${index + 1} did not return an ETag`);
completed.push({ partNumber: index + 1, etag: partEtag.replaceAll('"', "") });
offset = end;
}
const xml = `<CompleteMultipartUpload>${completed.map((part) => `<Part><PartNumber>${part.partNumber}</PartNumber><ETag>"${part.etag}"</ETag></Part>`).join("")}</CompleteMultipartUpload>`;
const result = await worker.fetch(await signed(`/test-bucket/big.bin?uploadId=${encodeURIComponent(uploadId)}`, { method: "POST", body: xml }), ENV, CTX);
expect(result.status).toBe(200);
expect(await result.text()).toMatch(/<ETag>"[0-9a-f]{32}"<\/ETag>/);
const stored = [...drive.files.values()].find((file) => file.name === "big.bin")!.data;
const storedFile = [...drive.files.values()].find((file) => file.name === "big.bin");
expect(storedFile).toBeDefined();
if (!storedFile) throw new Error("Completed multipart object was not stored");
const stored = storedFile.data;
expect(stored.byteLength).toBe(source.byteLength);
expect(fakeMd5(stored)).toBe(fakeMd5(source));
});
@@ -317,13 +372,13 @@ describe("S3 compatibility", () => {
CTX,
);
expect(response.status).toBe(200);
expect([...drive.files.values()].find((file) => file.name === "chunked.bin")!.data).toEqual(source);
expect([...drive.files.values()].find((file) => file.name === "chunked.bin")?.data).toEqual(source);
});
it("supports an empty PutObject", async () => {
const response = await worker.fetch(await signed("/test-bucket/empty", { method: "PUT", body: new Uint8Array() }), ENV, CTX);
expect(response.status).toBe(200);
expect([...drive.files.values()].find((file) => file.name === "empty")!.data.byteLength).toBe(0);
expect([...drive.files.values()].find((file) => file.name === "empty")?.data.byteLength).toBe(0);
});
it("lists nested keys under a prefix, as CommonPrefixes with a delimiter and recursively without one", async () => {
+2
View File
@@ -16,6 +16,8 @@ export default defineConfig({
ALLOWED_BUCKETS: "test-bucket,empty-bucket,my-bucket",
ALLOW_MULTIPART: "true",
ETAG_STYLE: "md5",
CORS_ALLOWED_ORIGINS: "http://localhost:5173",
ENABLE_DOCS: "true",
},
},
}),
+622 -1100
View File
File diff suppressed because it is too large Load Diff
+10 -4
View File
@@ -9,8 +9,16 @@
"compatibility_date": "2025-09-27",
"vars": {
"ALLOW_MULTIPART": "true",
"ETAG_STYLE": "md5"
"ETAG_STYLE": "md5",
"CORS_ALLOWED_ORIGINS": "",
"ENABLE_DOCS": "true"
},
"rules": [
{
"type": "Text",
"globs": ["docs/openapi.yaml"]
}
],
"observability": {
"enabled": true
},
@@ -25,9 +33,7 @@
"migrations": [
{
"tag": "v1",
"new_sqlite_classes": [
"MultipartUploadDO"
]
"new_sqlite_classes": ["MultipartUploadDO"]
}
],
"kv_namespaces": [