feat: add the openapi docs

This commit is contained in:
2026-08-16 16:43:21 +07:00
parent 92d4da14ed
commit a235040be7
27 changed files with 3558 additions and 2837 deletions
+27
View File
@@ -0,0 +1,27 @@
# Local-only Cloudflare Worker bindings. Copy to .dev.vars and replace each value.
# Secrets must never be committed.
# AUTH_KV, FOLDER_CACHE, and MPU are configured bindings in wrangler.jsonc, not .dev.vars values.
# S3 Signature V4 credentials held by CLI clients or your BFF.
ACCESS_KEY=replace-with-access-key-id
SECRET_KEY=replace-with-long-random-secret
REGION=auto
# Google OAuth credentials with Drive API access.
GOOGLE_CLIENT_ID=replace-with-google-oauth-client-id
GOOGLE_CLIENT_SECRET=replace-with-google-oauth-client-secret
GOOGLE_REFRESH_TOKEN=replace-with-google-refresh-token
# Comma-separated, exact bucket names. Unset denies every bucket.
ALLOWED_BUCKETS=assets
# Optional comma-separated subset of ALLOWED_BUCKETS that permits unsigned GET/HEAD.
PUBLIC_READ_BUCKETS=
# Durable Object multipart uploads and ETag result behavior.
ALLOW_MULTIPART=true
ETAG_STYLE=md5
# Browser CORS: comma-separated exact origins or *. Leave empty to disable CORS.
CORS_ALLOWED_ORIGINS=http://localhost:5173
# Set false to disable /docs and /openapi.yaml.
ENABLE_DOCS=true
+15 -2
View File
@@ -3,6 +3,10 @@ I.R.I.S. (Integrated Reliable Interop Storage)
Use Cloudflare Workers to turn your Google Drive into S3 object storage at no extra cost. Use Cloudflare Workers to turn your Google Drive into S3 object storage at no extra cost.
## API documentation
With `ENABLE_DOCS` left enabled (the default), each deployment serves an interactive API reference at `/docs` and the raw OpenAPI 3.1 document at `/openapi.yaml`. Frontend-specific setup, authentication details, limitations, and runnable examples live in [`docs/`](./docs/README.md).
## About ## About
This is a Workers script that converts the Google Drive API into an S3-compatible API. Turn your Google Drive into object storage at no extra cost! This is a Workers script that converts the Google Drive API into an S3-compatible API. Turn your Google Drive into object storage at no extra cost!
@@ -74,6 +78,8 @@ https://developers.cloudflare.com/workers/configuration/secrets/#via-the-dashboa
| `GOOGLE_CLIENT_ID`, `GOOGLE_CLIENT_SECRET`, `GOOGLE_REFRESH_TOKEN` | Google API credentials obtained from rclone. | | `GOOGLE_CLIENT_ID`, `GOOGLE_CLIENT_SECRET`, `GOOGLE_REFRESH_TOKEN` | Google API credentials obtained from rclone. |
| `ALLOWED_BUCKETS` | Set the buckets allowed, separated by `,`. A directory with the bucket name will be created directly under Google Drive. | | `ALLOWED_BUCKETS` | Set the buckets allowed, separated by `,`. A directory with the bucket name will be created directly under Google Drive. |
| `PUBLIC_READ_BUCKETS` | *(Optional)* Buckets that allow unauthenticated GET/HEAD access without signature, separated by `,`. Write operations (PUT/POST/DELETE) still require authentication. Must be a subset of `ALLOWED_BUCKETS`. | | `PUBLIC_READ_BUCKETS` | *(Optional)* Buckets that allow unauthenticated GET/HEAD access without signature, separated by `,`. Write operations (PUT/POST/DELETE) still require authentication. Must be a subset of `ALLOWED_BUCKETS`. |
| `CORS_ALLOWED_ORIGINS` | *(Optional)* Comma-separated exact browser origins, or `*`. Unset emits no CORS headers. |
| `ENABLE_DOCS` | *(Optional)* Set to `false` to disable `/docs` and `/openapi.yaml`; enabled by default. |
### 4. Enable Multipart Uploads ### 4. Enable Multipart Uploads
@@ -95,5 +101,12 @@ Google Drive's free tier has 15 GB total storage, and Google applies a 750 GB da
### 5. CORS Configuration ### 5. CORS Configuration
If you need to configure CORS, set up your own domain for Workers and use Cloudflare's Response Header Transform Rules to add the necessary headers. I.R.I.S. provides native, deny-by-default CORS handling. Set `CORS_ALLOWED_ORIGINS` to a comma-separated list of exact origins:
https://developers.cloudflare.com/rules/transform/response-header-modification/
```ini
CORS_ALLOWED_ORIGINS=https://app.example.com,http://localhost:5173
```
Use `*` only for a public, credential-free integration. Leave the value unset to emit no CORS headers, preserving CLI-only behavior. Allowed origins receive preflight support for S3 methods and the browser-readable response headers `ETag`, `Content-Range`, `Content-Length`, `Last-Modified`, `Accept-Ranges`, and `x-amz-request-id`. Server-to-server clients do not need CORS: requests without an `Origin` retain their normal S3 response and receive no `Access-Control-*` headers. Exact origin allow-lists add the harmless `Vary: Origin` response header for cache correctness.
For browser uploads, keep `SECRET_KEY` in a BFF and give the browser short-lived presigned URLs. See the [frontend integration guide](./docs/integration-guide.md).
+1 -1
View File
@@ -7,7 +7,7 @@
}, },
"files": { "files": {
"ignoreUnknown": true, "ignoreUnknown": true,
"includes": ["**", "!node_modules", "!.next", "!dist", "!build", "!src/app/globals.css"] "includes": ["**", "!node_modules", "!.next", "!dist", "!build", "!src/app/globals.css", "!worker-configuration.d.ts"]
}, },
"formatter": { "formatter": {
"lineWidth": 320, "lineWidth": 320,
+29
View File
@@ -0,0 +1,29 @@
# I.R.I.S. frontend integration docs
I.R.I.S. exposes a focused S3-compatible API on top of Google Drive. The live API reference is available from every enabled Worker deployment at [`/docs`](/docs), with the source OpenAPI document at [`/openapi.yaml`](/openapi.yaml).
## Start here
1. Configure the Worker with `CORS_ALLOWED_ORIGINS` for every browser origin that will upload or download objects.
2. Keep `SECRET_KEY` in a backend-for-frontend (BFF), not in browser code.
3. Let the BFF issue short-lived presigned URLs for a constrained bucket/key/method.
4. Upload or download through the presigned URL from the browser.
| Document | Purpose |
| --- | --- |
| [Integration guide](./integration-guide.md) | Browser upload, multipart upload, listing, CORS, and recommended architecture. |
| [Authentication](./authentication.md) | The precise AWS Signature V4 behavior enforced by I.R.I.S. |
| [Limitations](./limitations.md) | Unsupported S3 features and Google Drive operational limits. |
| [`examples/bff-presign.ts`](./examples/bff-presign.ts) | A Workers BFF endpoint that creates short-lived PUT URLs. |
| [`examples/browser-upload.ts`](./examples/browser-upload.ts) | Single object upload from a browser. |
| [`examples/browser-multipart.ts`](./examples/browser-multipart.ts) | Strictly sequential browser multipart upload. |
## Browser security model
```text
Browser ── POST /presign ──> BFF (holds SECRET_KEY)
│ │
└──── presigned S3 request ────┴──> I.R.I.S. Worker ──> Google Drive
```
The browser never receives `SECRET_KEY`. Presigned URLs must be short lived and scoped to the single object operation the browser needs.
+39
View File
@@ -0,0 +1,39 @@
# Authentication and Signature V4
I.R.I.S. accepts AWS Signature Version 4 in either form:
- an `Authorization: AWS4-HMAC-SHA256 …` header with `x-amz-date`; or
- a presigned request with `X-Amz-Algorithm`, `X-Amz-Credential`, `X-Amz-Date`, `X-Amz-SignedHeaders`, and `X-Amz-Signature` query parameters.
The `Credential` access-key ID must equal the Worker's `ACCESS_KEY`; the signing key is derived from `SECRET_KEY`, `REGION`, service `s3`, and the `YYYYMMDD` request date.
## Presigned URL expiry
Query-authenticated requests require `X-Amz-Expires` from 1 through 604800 seconds (seven days). The Worker returns `403 AccessDenied` with `Request has expired` when it is missing, invalid, out of range, or its signed timestamp plus expiry is in the past.
BFFs should normally use a much shorter period such as five minutes.
## Header-authenticated requests
The Worker requires a valid `x-amz-date` and rejects a request when the server time differs from it by more than 15 minutes. This returns `403 RequestTimeTooSkewed`.
## Canonical request behavior
The canonical request is built from:
1. method;
2. URL path;
3. sorted query string (excluding `X-Amz-Signature` for presigned URLs);
4. the signed-header list and normalized values;
5. the signed-header list; and
6. `x-amz-content-sha256`, defaulting to `UNSIGNED-PAYLOAD`.
I.R.I.S. deliberately canonicalizes a signed `accept-encoding` header to `identity`. Cloudflare can rewrite the received value at the edge; S3 SDKs that sign this header use `identity` for this reason. Browser code must not sign `accept-encoding` because browser networking controls it.
Payload hashes are **not** verified. Browser and BFF clients should use `x-amz-content-sha256: UNSIGNED-PAYLOAD`; this is a deliberate streaming limitation, not an integrity guarantee.
## Public-read buckets
Buckets listed in `PUBLIC_READ_BUCKETS` permit unsigned `GET` and `HEAD` requests. All write operations still require valid Signature V4 authentication. `PUBLIC_READ_BUCKETS` must be a subset of `ALLOWED_BUCKETS`.
For a tested signing reference, see the `signed()` helper in [`test/s3.test.ts`](../test/s3.test.ts).
+95
View File
@@ -0,0 +1,95 @@
import { AbortMultipartUploadCommand, CompleteMultipartUploadCommand, CreateMultipartUploadCommand, PutObjectCommand, S3Client, UploadPartCommand } from "@aws-sdk/client-s3";
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
interface Env {
IRIS_ENDPOINT: string;
IRIS_ACCESS_KEY: string;
IRIS_SECRET_KEY: string;
IRIS_REGION: string;
IRIS_BUCKET: string;
}
interface PresignRequest {
key: string;
contentType?: string;
operation?: "put" | "createMultipart" | "uploadPart" | "completeMultipart" | "abortMultipart";
uploadId?: string;
partNumber?: number;
parts?: Array<{ ETag: string; PartNumber: number }>;
}
function escapeXml(value: string): string {
return value.replace(/&/g, "&amp;").replace(/</g, "&lt;").replace(/>/g, "&gt;").replace(/"/g, "&quot;").replace(/'/g, "&apos;");
}
function completionXml(parts: Array<{ ETag: string; PartNumber: number }> | undefined): string | null {
if (!parts?.length || parts.some((part, index) => !Number.isInteger(part.PartNumber) || part.PartNumber !== index + 1 || typeof part.ETag !== "string" || part.ETag.length === 0)) return null;
return `<CompleteMultipartUpload>${parts.map((part) => `<Part><PartNumber>${part.PartNumber}</PartNumber><ETag>${escapeXml(part.ETag)}</ETag></Part>`).join("")}</CompleteMultipartUpload>`;
}
function json(data: unknown, status = 200): Response {
return Response.json(data, { status, headers: { "Cache-Control": "no-store" } });
}
function s3(env: Env): S3Client {
return new S3Client({
endpoint: env.IRIS_ENDPOINT,
region: env.IRIS_REGION,
forcePathStyle: true,
credentials: { accessKeyId: env.IRIS_ACCESS_KEY, secretAccessKey: env.IRIS_SECRET_KEY },
});
}
function validKey(key: unknown): key is string {
return typeof key === "string" && key.length > 0 && key.length <= 1024 && !key.startsWith("/") && !key.split("/").includes("..");
}
export default {
async fetch(request: Request, env: Env): Promise<Response> {
if (request.method !== "POST" || new URL(request.url).pathname !== "/presign") return new Response("Not found", { status: 404 });
let input: PresignRequest;
try {
input = await request.json<PresignRequest>();
} catch {
return json({ error: "Expected JSON body" }, 400);
}
if (!validKey(input.key)) return json({ error: "Invalid object key" }, 400);
const client = s3(env);
const operation = input.operation ?? "put";
const expiresIn = 5 * 60;
if (operation === "put") {
const contentType = input.contentType || "application/octet-stream";
const url = await getSignedUrl(client, new PutObjectCommand({ Bucket: env.IRIS_BUCKET, Key: input.key, ContentType: contentType }), { expiresIn });
return json({ url, method: "PUT", headers: { "Content-Type": contentType }, expiresIn });
}
if (operation === "createMultipart") {
const contentType = input.contentType || "application/octet-stream";
const url = await getSignedUrl(client, new CreateMultipartUploadCommand({ Bucket: env.IRIS_BUCKET, Key: input.key, ContentType: contentType }), { expiresIn });
return json({ url, method: "POST", headers: { "Content-Type": contentType }, expiresIn });
}
if (!input.uploadId) return json({ error: "uploadId is required" }, 400);
if (operation === "uploadPart") {
const partNumber = input.partNumber;
if (!Number.isInteger(partNumber) || partNumber === undefined || partNumber < 1 || partNumber > 10_000) return json({ error: "partNumber must be 1 through 10000" }, 400);
const url = await getSignedUrl(client, new UploadPartCommand({ Bucket: env.IRIS_BUCKET, Key: input.key, UploadId: input.uploadId, PartNumber: partNumber }), { expiresIn });
return json({ url, method: "PUT", expiresIn });
}
if (operation === "completeMultipart") {
const body = completionXml(input.parts);
if (!body) return json({ error: "parts must have sequential PartNumber values beginning at 1 and non-empty ETag values" }, 400);
const url = await getSignedUrl(client, new CompleteMultipartUploadCommand({ Bucket: env.IRIS_BUCKET, Key: input.key, UploadId: input.uploadId, MultipartUpload: { Parts: input.parts } }), { expiresIn });
return json({ url, method: "POST", headers: { "Content-Type": "application/xml" }, body, expiresIn });
}
if (operation === "abortMultipart") {
const url = await getSignedUrl(client, new AbortMultipartUploadCommand({ Bucket: env.IRIS_BUCKET, Key: input.key, UploadId: input.uploadId }), { expiresIn });
return json({ url, method: "DELETE", expiresIn });
}
return json({ error: "Unsupported operation" }, 400);
},
} satisfies ExportedHandler<Env>;
+48
View File
@@ -0,0 +1,48 @@
interface PresignResponse {
url: string;
method: string;
headers?: Record<string, string>;
body?: string;
}
async function presign(body: Record<string, unknown>): Promise<PresignResponse> {
const response = await fetch("/presign", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(body),
});
if (!response.ok) throw new Error(await response.text());
return response.json<PresignResponse>();
}
function uploadIdFromXml(xml: string): string {
const value = /<UploadId>([^<]+)<\/UploadId>/.exec(xml)?.[1];
if (!value) throw new Error("CreateMultipartUpload did not return UploadId");
return value;
}
/** Uploads parts in order. I.R.I.S. does not allow parallel or out-of-order part uploads. */
export async function uploadLargeFile(file: File, key: string, partSize = 16 * 1024 * 1024): Promise<void> {
const contentType = file.type || "application/octet-stream";
const create = await presign({ key, contentType, operation: "createMultipart" });
const createResponse = await fetch(create.url, { method: create.method, headers: create.headers });
if (!createResponse.ok) throw new Error(await createResponse.text());
const uploadId = uploadIdFromXml(await createResponse.text());
const parts: Array<{ ETag: string; PartNumber: number }> = [];
for (let index = 0, offset = 0; offset < file.size; index++, offset += partSize) {
const partNumber = index + 1;
const body = file.slice(offset, Math.min(offset + partSize, file.size));
const signed = await presign({ key, uploadId, partNumber, operation: "uploadPart" });
const response = await fetch(signed.url, { method: signed.method, body });
if (!response.ok) throw new Error(`Part ${partNumber} failed: ${await response.text()}`);
const etag = response.headers.get("ETag");
if (!etag) throw new Error(`Part ${partNumber} did not return ETag; check CORS_ALLOWED_ORIGINS`);
parts.push({ ETag: etag, PartNumber: partNumber });
}
const complete = await presign({ key, uploadId, parts, operation: "completeMultipart" });
if (!complete.body) throw new Error("BFF did not return the completion XML body");
const completeResponse = await fetch(complete.url, { method: complete.method, headers: complete.headers, body: complete.body });
if (!completeResponse.ok) throw new Error(await completeResponse.text());
}
+29
View File
@@ -0,0 +1,29 @@
interface PresignedPut {
url: string;
method: "PUT";
headers: { "Content-Type": string };
expiresIn: number;
}
async function getPresignedPut(key: string, contentType: string): Promise<PresignedPut> {
const response = await fetch("/presign", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ key, contentType, operation: "put" }),
});
if (!response.ok) throw new Error(await response.text());
return response.json<PresignedPut>();
}
export async function uploadFile(file: File, key: string): Promise<string | null> {
const signed = await getPresignedPut(key, file.type || "application/octet-stream");
const response = await fetch(signed.url, {
method: signed.method,
headers: signed.headers,
body: file,
});
if (!response.ok) throw new Error(`Upload failed: ${await response.text()}`);
// Requires CORS_ALLOWED_ORIGINS to include this page's exact origin.
return response.headers.get("ETag");
}
+98
View File
@@ -0,0 +1,98 @@
# Frontend integration guide
## Architecture
Use a backend-for-frontend (BFF) as the trust boundary:
```text
Browser → BFF (holds ACCESS_KEY + SECRET_KEY, issues presigned URLs) → I.R.I.S. Worker → Google Drive
```
Never put `SECRET_KEY` in browser code, a static site, or a public environment variable. The browser receives only a short-lived presigned URL for one method and object key.
## Enable browser access
Configure the Worker with an explicit allow-list of web origins:
```ini
CORS_ALLOWED_ORIGINS=https://app.example.com,http://localhost:5173
```
Set it to `*` only for a genuinely public, credential-free integration. Leaving it unset emits no CORS headers. I.R.I.S. exposes `ETag`, `Content-Range`, `Content-Length`, `Last-Modified`, `Accept-Ranges`, and `x-amz-request-id`, so browser code can inspect the headers that form the effective S3 response payload.
### Server-to-server compatibility
CORS does not restrict non-browser clients. `Access-Control-*` headers are emitted only when a request has an allowed `Origin`; a CLI, Dokploy, rclone, aws-cli, or other service request with no `Origin` keeps its normal S3 status and body. With an exact origin allow-list, responses also include `Vary: Origin`, including requests without an `Origin`, so shared caches cannot replay a no-origin object response to a browser. S3 clients ignore this cache-control metadata.
An `OPTIONS` request with no `Origin`, or a preflight from a disallowed origin, receives the same bare `204` as a CORS-disabled Worker. A disallowed `Origin` never changes the underlying S3 result—it simply receives no `Access-Control-*` headers.
## Presign in the BFF
Use `@aws-sdk/s3-request-presigner` and send a limited expiry. I.R.I.S. validates `X-Amz-Expires` and rejects URLs after their signing time plus that value.
```ts
import { PutObjectCommand, S3Client } from "@aws-sdk/client-s3";
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
const client = new S3Client({
endpoint: env.IRIS_ENDPOINT,
region: env.IRIS_REGION,
credentials: {
accessKeyId: env.IRIS_ACCESS_KEY,
secretAccessKey: env.IRIS_SECRET_KEY,
},
forcePathStyle: true,
});
const url = await getSignedUrl(
client,
new PutObjectCommand({
Bucket: "assets",
Key: "uploads/avatar.png",
ContentType: "image/png",
}),
{ expiresIn: 300 },
);
```
Keep the `ContentType` in the command and send precisely that same `Content-Type` from the browser. It becomes the Drive `mimeType`. A non-simple content type causes a preflight request, which is why the Worker CORS configuration is necessary.
## Upload one object
Use `Blob`, `File`, `ArrayBuffer`, or a typed array as the body. This lets the browser provide a known body length. Do **not** use a `ReadableStream` unless the BFF signs `x-amz-decoded-content-length` and the browser sends it: multipart part uploads require `Content-Length` or `x-amz-decoded-content-length`.
```ts
const response = await fetch(presignedUrl, {
method: "PUT",
headers: { "Content-Type": file.type || "application/octet-stream" },
body: file,
});
if (!response.ok) throw new Error(await response.text());
const etag = response.headers.get("ETag");
```
`ETag` is readable only when the request `Origin` is configured in `CORS_ALLOWED_ORIGINS`.
## Multipart upload
Cloudflare request limits mean a single PUT is best kept below roughly 100 MB. For larger files, have the BFF initiate the upload, issue a presigned URL for each part, and complete the upload after the browser returns all ETags.
Use 16 MB parts as a practical baseline. Upload parts **strictly sequentially starting at 1**. I.R.I.S. waits up to 20 seconds for an earlier part; out-of-order work then receives `503 SlowDown`. Requests more than 64 part numbers ahead are rejected immediately.
1. BFF signs `POST /bucket/key?uploads` and starts the multipart upload.
2. BFF issues a signed `PUT` URL for part 1; browser uploads it and records the `ETag`.
3. Repeat one part at a time for parts 2 through $n$.
4. BFF signs `POST /bucket/key?uploadId=…`; browser sends the `CompleteMultipartUpload` XML body containing the sequential part numbers and ETags.
See [`examples/browser-multipart.ts`](./examples/browser-multipart.ts) for a complete browser-side loop.
## Build a file browser
List with `prefix` and `delimiter=/`:
```text
GET /assets?prefix=photos/&delimiter=/
```
Files appear in `Contents`; immediate child folders appear as `CommonPrefixes`. Google Drive directories are physical folders, not zero-byte marker objects. Read [limitations](./limitations.md) before designing pagination or rename/move features.
+35
View File
@@ -0,0 +1,35 @@
# Limitations
I.R.I.S. is a narrow S3 compatibility layer. Design frontend features around the behavior below.
## Listing is not pageable
`max-keys`, `continuation-token`, and `marker` are ignored. I.R.I.S. scans at most 5000 Google Drive nodes and then sets `IsTruncated=true`; it does not provide a continuation token, so additional results cannot be requested. The XML value `MaxKeys=1000` is a fixed compatibility value, not the real scan limit.
Use narrow prefixes and `delimiter=/` for a file browser. Do not build an infinite-scrolling full-drive browser.
## No server-side rename or move
`CopyObject` and `UploadPartCopy` return `501 NotImplemented`. There is no server-side rename or move because both require copying. Implement rename/move as download, re-upload under the target key, then delete the original—subject to bandwidth, time, and error handling.
## Unsupported S3 features
- `ListBuckets`, `CreateBucket`, and `DeleteBucket`
- batch `DeleteObjects`
- object versioning, ACLs, object tags, lifecycle rules, and `x-amz-meta-*` user metadata
- CopyObject and UploadPartCopy
- discovering active multipart uploads: `ListMultipartUploads` always reports an empty list
Bucket `?acl`, `?versioning`, and `?location` requests are currently treated as regular bucket listing requests rather than errors. Do not rely on this accidental compatibility.
## Directories and objects
Directories are physical Google Drive folders created from object-key path segments. I.R.I.S. does not create zero-byte directory marker objects. An empty prefix can therefore exist as a Drive folder without an S3 marker object.
## Multipart behavior
Parts must be uploaded strictly in consecutive order beginning at part 1. Parts are immutable after they have committed. An incomplete upload expires after 24 hours. `ETAG_STYLE=md5` returns Google Drive's file MD5 for completed multipart objects; `ETAG_STYLE=multipart` is available only for clients that require an S3-style composite ETag.
## Quotas and plan limits
Google Drive's free tier has 15 GB total storage and a 750 GB/day upload limit. Cloudflare applies request-size limits before a request reaches the Worker; use multipart uploads for large browser uploads.
+391
View File
@@ -0,0 +1,391 @@
openapi: 3.1.0
info:
title: I.R.I.S. Google Drive S3 API
version: 1.0.0
description: |
An intentionally small S3-compatible object-storage API backed by Google Drive.
Authenticate every non-public request with AWS Signature Version 4. Browser clients
should receive short-lived presigned URLs from a backend-for-frontend, never credentials.
servers:
- url: https://{worker-host}
variables:
worker-host:
default: your-worker.example.workers.dev
security:
- sigv4: []
tags:
- name: Objects
- name: Buckets
- name: Multipart uploads
paths:
/{bucket}:
parameters:
- $ref: '#/components/parameters/Bucket'
get:
tags: [Buckets]
operationId: listObjects
summary: List objects in a bucket
description: List objects recursively, or use `delimiter=/` to return immediate folders as `CommonPrefixes`.
parameters:
- $ref: '#/components/parameters/Prefix'
- $ref: '#/components/parameters/Delimiter'
responses:
'200':
description: Object listing.
content:
application/xml:
schema: { $ref: '#/components/schemas/ListBucketResult' }
'403': { $ref: '#/components/responses/AccessDenied' }
'405': { $ref: '#/components/responses/MethodNotAllowed' }
head:
tags: [Buckets]
operationId: headBucket
summary: Check bucket access
responses:
'200': { description: The bucket is configured and accessible. }
'403': { $ref: '#/components/responses/AccessDenied' }
'405': { $ref: '#/components/responses/MethodNotAllowed' }
/{bucket}?uploads:
parameters:
- $ref: '#/components/parameters/Bucket'
get:
tags: [Multipart uploads]
operationId: listMultipartUploads
summary: List multipart uploads
description: Always returns an empty list; active multipart uploads cannot be enumerated.
responses:
'200':
description: Empty multipart upload listing.
content:
application/xml:
schema: { $ref: '#/components/schemas/ListMultipartUploadsResult' }
'403': { $ref: '#/components/responses/AccessDenied' }
/{bucket}/{key}:
parameters:
- $ref: '#/components/parameters/Bucket'
- $ref: '#/components/parameters/Key'
put:
tags: [Objects]
operationId: putObject
summary: Create or replace an object
parameters:
- $ref: '#/components/parameters/ContentType'
requestBody:
required: true
content:
application/octet-stream:
schema: { type: string, format: binary }
'*/*':
schema: { type: string, format: binary }
responses:
'200':
description: Object uploaded.
headers:
ETag: { $ref: '#/components/headers/ETag' }
'403': { $ref: '#/components/responses/AccessDenied' }
'501': { $ref: '#/components/responses/NotImplemented' }
get:
tags: [Objects]
operationId: getObject
summary: Download an object
parameters:
- name: Range
in: header
schema: { type: string, example: bytes=0-1023 }
responses:
'200':
description: Full object data.
headers:
ETag: { $ref: '#/components/headers/ETag' }
Content-Length: { $ref: '#/components/headers/ContentLength' }
Last-Modified: { $ref: '#/components/headers/LastModified' }
Accept-Ranges: { schema: { type: string, example: bytes } }
content:
application/octet-stream:
schema: { type: string, format: binary }
'206':
description: Requested byte range.
headers:
Content-Range: { schema: { type: string, example: bytes 0-1023/4096 } }
Content-Length: { $ref: '#/components/headers/ContentLength' }
content:
application/octet-stream:
schema: { type: string, format: binary }
'403': { $ref: '#/components/responses/AccessDenied' }
'404': { $ref: '#/components/responses/NoSuchKey' }
head:
tags: [Objects]
operationId: headObject
summary: Get object metadata
responses:
'200':
description: Object metadata in headers.
headers:
ETag: { $ref: '#/components/headers/ETag' }
Content-Length: { $ref: '#/components/headers/ContentLength' }
Last-Modified: { $ref: '#/components/headers/LastModified' }
Accept-Ranges: { schema: { type: string, example: bytes } }
'403': { $ref: '#/components/responses/AccessDenied' }
'404': { $ref: '#/components/responses/NoSuchKey' }
delete:
tags: [Objects]
operationId: deleteObject
summary: Delete an object
responses:
'204': { description: Object deleted. }
'403': { $ref: '#/components/responses/AccessDenied' }
'404': { $ref: '#/components/responses/NoSuchKey' }
/{bucket}/{key}?uploads:
parameters:
- $ref: '#/components/parameters/Bucket'
- $ref: '#/components/parameters/Key'
post:
tags: [Multipart uploads]
operationId: createMultipartUpload
summary: Initiate a multipart upload
parameters:
- $ref: '#/components/parameters/ContentType'
responses:
'200':
description: Upload initiated.
content:
application/xml:
schema: { $ref: '#/components/schemas/InitiateMultipartUploadResult' }
'403': { $ref: '#/components/responses/AccessDenied' }
'501': { $ref: '#/components/responses/NotImplemented' }
/{bucket}/{key}?uploadId={uploadId}&partNumber={partNumber}:
parameters:
- $ref: '#/components/parameters/Bucket'
- $ref: '#/components/parameters/Key'
- $ref: '#/components/parameters/UploadId'
- $ref: '#/components/parameters/PartNumber'
put:
tags: [Multipart uploads]
operationId: uploadPart
summary: Upload one multipart part
description: Parts must arrive strictly sequentially beginning with part number 1.
requestBody:
required: true
content:
application/octet-stream:
schema: { type: string, format: binary }
responses:
'200':
description: Part accepted.
headers:
ETag: { $ref: '#/components/headers/ETag' }
'400': { $ref: '#/components/responses/InvalidArgument' }
'403': { $ref: '#/components/responses/AccessDenied' }
'404': { $ref: '#/components/responses/NoSuchUpload' }
'503': { $ref: '#/components/responses/SlowDown' }
/{bucket}/{key}?uploadId={uploadId}:
parameters:
- $ref: '#/components/parameters/Bucket'
- $ref: '#/components/parameters/Key'
- $ref: '#/components/parameters/UploadId'
post:
tags: [Multipart uploads]
operationId: completeMultipartUpload
summary: Complete a multipart upload
requestBody:
required: true
content:
application/xml:
schema: { $ref: '#/components/schemas/CompleteMultipartUpload' }
responses:
'200':
description: Upload completed.
content:
application/xml:
schema: { $ref: '#/components/schemas/CompleteMultipartUploadResult' }
'400': { $ref: '#/components/responses/InvalidArgument' }
'403': { $ref: '#/components/responses/AccessDenied' }
'404': { $ref: '#/components/responses/NoSuchUpload' }
get:
tags: [Multipart uploads]
operationId: listParts
summary: List uploaded parts
parameters:
- name: part-number-marker
in: query
schema: { type: integer, minimum: 0, default: 0 }
- name: max-parts
in: query
schema: { type: integer, minimum: 1, maximum: 1000, default: 1000 }
responses:
'200':
description: Current part list.
content:
application/xml:
schema: { $ref: '#/components/schemas/ListPartsResult' }
'400': { $ref: '#/components/responses/InvalidArgument' }
'403': { $ref: '#/components/responses/AccessDenied' }
'404': { $ref: '#/components/responses/NoSuchUpload' }
delete:
tags: [Multipart uploads]
operationId: abortMultipartUpload
summary: Abort a multipart upload
responses:
'204': { description: Upload aborted. }
'403': { $ref: '#/components/responses/AccessDenied' }
'404': { $ref: '#/components/responses/NoSuchUpload' }
components:
securitySchemes:
sigv4:
type: apiKey
in: header
name: Authorization
description: AWS Signature Version 4 header authentication or equivalent `X-Amz-*` presigned query parameters.
parameters:
Bucket:
name: bucket
in: path
required: true
schema: { type: string }
Key:
name: key
in: path
required: true
schema: { type: string }
Prefix:
name: prefix
in: query
schema: { type: string, default: '' }
Delimiter:
name: delimiter
in: query
schema: { type: string, example: / }
UploadId:
name: uploadId
in: query
required: true
schema: { type: string }
PartNumber:
name: partNumber
in: query
required: true
schema: { type: integer, minimum: 1, maximum: 10000 }
ContentType:
name: Content-Type
in: header
schema: { type: string, example: application/octet-stream }
headers:
ETag:
schema: { type: string, example: '"d41d8cd98f00b204e9800998ecf8427e"' }
ContentLength:
schema: { type: integer, minimum: 0 }
LastModified:
schema: { type: string, format: date-time }
responses:
AccessDenied:
description: Authentication failed, the access key is not recognized, the presigned URL expired, or bucket access is denied.
content: { application/xml: { schema: { $ref: '#/components/schemas/S3Error' } } }
NoSuchKey:
description: The object does not exist.
content: { application/xml: { schema: { $ref: '#/components/schemas/S3Error' } } }
NoSuchUpload:
description: The multipart upload does not exist.
content: { application/xml: { schema: { $ref: '#/components/schemas/S3Error' } } }
InvalidArgument:
description: A required parameter or body is invalid.
content: { application/xml: { schema: { $ref: '#/components/schemas/S3Error' } } }
NotImplemented:
description: This S3 operation is not supported.
content: { application/xml: { schema: { $ref: '#/components/schemas/S3Error' } } }
SlowDown:
description: Parts are out of sequence or the upload is busy; retry later.
content: { application/xml: { schema: { $ref: '#/components/schemas/S3Error' } } }
MethodNotAllowed:
description: The method/path combination is unsupported.
content: { application/xml: { schema: { $ref: '#/components/schemas/S3Error' } } }
schemas:
S3Error:
type: object
xml: { name: Error }
required: [Code, Message, RequestId]
properties:
Code: { type: string, xml: { name: Code } }
Message: { type: string, xml: { name: Message } }
Resource: { type: string, xml: { name: Resource } }
RequestId: { type: string, xml: { name: RequestId } }
Object:
type: object
xml: { name: Contents }
properties:
Key: { type: string }
LastModified: { type: string, format: date-time }
ETag: { type: string }
Size: { type: integer }
StorageClass: { type: string, example: STANDARD }
ListBucketResult:
type: object
xml: { name: ListBucketResult, namespace: 'http://s3.amazonaws.com/doc/2006-03-01/' }
properties:
Name: { type: string }
Prefix: { type: string }
Delimiter: { type: string }
MaxKeys: { type: integer, example: 1000 }
IsTruncated: { type: boolean }
Contents: { type: array, items: { $ref: '#/components/schemas/Object' }, xml: { wrapped: false } }
InitiateMultipartUploadResult:
type: object
xml: { name: InitiateMultipartUploadResult, namespace: 'http://s3.amazonaws.com/doc/2006-03-01/' }
required: [Bucket, Key, UploadId]
properties:
Bucket: { type: string }
Key: { type: string }
UploadId: { type: string }
CompleteMultipartUpload:
type: object
xml: { name: CompleteMultipartUpload }
required: [Part]
properties:
Part:
type: array
minItems: 1
xml: { wrapped: false }
items:
type: object
xml: { name: Part }
required: [PartNumber, ETag]
properties:
PartNumber: { type: integer }
ETag: { type: string }
CompleteMultipartUploadResult:
type: object
xml: { name: CompleteMultipartUploadResult, namespace: 'http://s3.amazonaws.com/doc/2006-03-01/' }
properties:
Location: { type: string }
Bucket: { type: string }
Key: { type: string }
ETag: { type: string }
ListPartsResult:
type: object
xml: { name: ListPartsResult, namespace: 'http://s3.amazonaws.com/doc/2006-03-01/' }
properties:
Bucket: { type: string }
Key: { type: string }
UploadId: { type: string }
NextPartNumberMarker: { type: integer }
IsTruncated: { type: boolean }
Part:
type: array
xml: { wrapped: false }
items:
type: object
properties:
PartNumber: { type: integer }
LastModified: { type: string, format: date-time }
ETag: { type: string }
Size: { type: integer }
ListMultipartUploadsResult:
type: object
xml: { name: ListMultipartUploadsResult, namespace: 'http://s3.amazonaws.com/doc/2006-03-01/' }
properties:
Bucket: { type: string }
KeyMarker: { type: string }
UploadIdMarker: { type: string }
NextKeyMarker: { type: string }
NextUploadIdMarker: { type: string }
MaxUploads: { type: integer, example: 1000 }
IsTruncated: { type: boolean, example: false }
+2 -5
View File
@@ -20,9 +20,9 @@
// Optional: --port 8976 (must not be in use; Desktop-app clients accept any // Optional: --port 8976 (must not be in use; Desktop-app clients accept any
// loopback port automatically, no need to register it in Google Cloud Console). // loopback port automatically, no need to register it in Google Cloud Console).
import { exec } from "node:child_process";
import { randomBytes } from "node:crypto"; import { randomBytes } from "node:crypto";
import http from "node:http"; import http from "node:http";
import { exec } from "node:child_process";
import { URL } from "node:url"; import { URL } from "node:url";
const SCOPE = "https://www.googleapis.com/auth/drive"; const SCOPE = "https://www.googleapis.com/auth/drive";
@@ -137,10 +137,7 @@ async function main() {
process.exit(1); process.exit(1);
} }
if (!tokenData.refresh_token) { if (!tokenData.refresh_token) {
console.error( console.error("No refresh_token was returned. This usually means the account already has an active grant.\n" + "Revoke it at https://myaccount.google.com/permissions and re-run this script.");
"No refresh_token was returned. This usually means the account already has an active grant.\n" +
"Revoke it at https://myaccount.google.com/permissions and re-run this script.",
);
process.exit(1); process.exit(1);
} }
+56 -6
View File
@@ -1,5 +1,11 @@
import type { S3ErrorCode } from "./s3-errors";
import type { Env } from "./types"; import type { Env } from "./types";
const MAX_PRESIGN_EXPIRY_SECONDS = 7 * 24 * 60 * 60;
const MAX_CLOCK_SKEW_MS = 15 * 60 * 1000;
export type VerifyResult = { ok: true } | { ok: false; code: S3ErrorCode; message?: string };
function encodeRFC3986(str: string): string { function encodeRFC3986(str: string): string {
return encodeURIComponent(str).replace(/[!'()*]/g, (c) => `%${c.charCodeAt(0).toString(16).toUpperCase()}`); return encodeURIComponent(str).replace(/[!'()*]/g, (c) => `%${c.charCodeAt(0).toString(16).toUpperCase()}`);
} }
@@ -84,8 +90,35 @@ async function createCanonicalRequest(request: Request, isQueryAuth: boolean): P
return [method, canonicalUri, params, canonicalHeaders, signedHeaders, payloadHash].join("\n"); return [method, canonicalUri, params, canonicalHeaders, signedHeaders, payloadHash].join("\n");
} }
function parseAmzDate(datetime: string): number | null {
const match = /^(\d{4})(\d{2})(\d{2})T(\d{2})(\d{2})(\d{2})Z$/.exec(datetime);
if (!match) return null;
const [, year, month, day, hour, minute, second] = match;
const timestamp = Date.UTC(Number(year), Number(month) - 1, Number(day), Number(hour), Number(minute), Number(second));
const parsed = new Date(timestamp);
if (parsed.getUTCFullYear() !== Number(year) || parsed.getUTCMonth() !== Number(month) - 1 || parsed.getUTCDate() !== Number(day) || parsed.getUTCHours() !== Number(hour) || parsed.getUTCMinutes() !== Number(minute) || parsed.getUTCSeconds() !== Number(second)) {
return null;
}
return timestamp;
}
function constantTimeEqual(left: string, right: string): boolean {
if (left.length !== right.length) return false;
let mismatch = 0;
for (let index = 0; index < left.length; index++) {
mismatch |= left.charCodeAt(index) ^ right.charCodeAt(index);
}
return mismatch === 0;
}
function signatureMismatch(): VerifyResult {
return { ok: false, code: "SignatureDoesNotMatch" };
}
/** Verifies an AWS Signature V4 signature carried in either the Authorization header or presigned query params. */ /** Verifies an AWS Signature V4 signature carried in either the Authorization header or presigned query params. */
export async function verifySignature(request: Request, env: Env): Promise<boolean> { export async function verifySignature(request: Request, env: Env): Promise<VerifyResult> {
const url = new URL(request.url); const url = new URL(request.url);
const headers = request.headers; const headers = request.headers;
@@ -99,15 +132,32 @@ export async function verifySignature(request: Request, env: Env): Promise<boole
algorithm = authHeader.split(" ")[0]; algorithm = authHeader.split(" ")[0];
} }
if (!algorithm?.includes("AWS4-HMAC-SHA256")) { if (algorithm !== "AWS4-HMAC-SHA256") {
return false; return signatureMismatch();
} }
const datetime = (isQueryAuth ? url.searchParams.get("X-Amz-Date") : headers.get("x-amz-date")) ?? ""; const datetime = (isQueryAuth ? url.searchParams.get("X-Amz-Date") : headers.get("x-amz-date")) ?? "";
if (!datetime) return isQueryAuth ? { ok: false, code: "AccessDenied", message: "Request has expired" } : signatureMismatch();
const signedAt = parseAmzDate(datetime);
if (signedAt === null) return isQueryAuth ? { ok: false, code: "AccessDenied", message: "Request has expired" } : signatureMismatch();
if (!datetime) return false; if (isQueryAuth) {
const expires = url.searchParams.get("X-Amz-Expires");
if (!expires || !/^\d+$/.test(expires)) return { ok: false, code: "AccessDenied", message: "Request has expired" };
const expiresIn = Number(expires);
if (!Number.isSafeInteger(expiresIn) || expiresIn < 1 || expiresIn > MAX_PRESIGN_EXPIRY_SECONDS || Date.now() > signedAt + expiresIn * 1000) {
return { ok: false, code: "AccessDenied", message: "Request has expired" };
}
} else if (Math.abs(Date.now() - signedAt) > MAX_CLOCK_SKEW_MS) {
return { ok: false, code: "RequestTimeTooSkewed" };
}
const date = datetime.substring(0, 8); const date = datetime.substring(0, 8);
const credential = isQueryAuth ? (url.searchParams.get("X-Amz-Credential") ?? "") : (/Credential=([^,\s]+)/.exec(headers.get("Authorization") ?? "")?.[1] ?? "");
const credentialParts = credential.split("/");
if (credentialParts.length !== 5 || credentialParts[0] !== env.ACCESS_KEY || credentialParts.slice(1).join("/") !== `${date}/${env.REGION}/s3/aws4_request`) {
return { ok: false, code: "AccessDenied" };
}
const canonicalRequest = await createCanonicalRequest(request, isQueryAuth); const canonicalRequest = await createCanonicalRequest(request, isQueryAuth);
const hashedCanonicalRequest = await sha256(canonicalRequest); const hashedCanonicalRequest = await sha256(canonicalRequest);
@@ -124,9 +174,9 @@ export async function verifySignature(request: Request, env: Env): Promise<boole
expectedSignature = url.searchParams.get("X-Amz-Signature") ?? ""; expectedSignature = url.searchParams.get("X-Amz-Signature") ?? "";
} else { } else {
const authHeader = headers.get("Authorization") ?? ""; const authHeader = headers.get("Authorization") ?? "";
const match = authHeader.match(/Signature=([a-f0-9]+)/); const match = authHeader.match(/Signature=([a-fA-F0-9]+)/);
expectedSignature = match ? match[1] : ""; expectedSignature = match ? match[1] : "";
} }
return signatureHex === expectedSignature; return constantTimeEqual(signatureHex, expectedSignature) ? { ok: true } : signatureMismatch();
} }
+69
View File
@@ -0,0 +1,69 @@
import type { Env } from "./types";
const ALLOWED_METHODS = "GET, HEAD, PUT, POST, DELETE, OPTIONS";
const DEFAULT_ALLOWED_HEADERS = "Authorization, Content-Type, Content-Length, Content-MD5, Content-Encoding, Range, x-amz-content-sha256, x-amz-copy-source, x-amz-date, x-amz-decoded-content-length, x-amz-mp-object-size, x-amz-security-token";
const EXPOSED_HEADERS = "ETag, Content-Range, Content-Length, Last-Modified, Accept-Ranges, x-amz-request-id";
function configuredOrigins(env: Env): string[] {
return (env.CORS_ALLOWED_ORIGINS ?? "")
.split(",")
.map((origin) => origin.trim())
.filter(Boolean);
}
function hasWildcardOrigin(env: Env): boolean {
return env.CORS_ALLOWED_ORIGINS?.trim() === "*";
}
function appendVary(headers: Headers, value: string): void {
const existing = headers.get("Vary");
if (!existing) {
headers.set("Vary", value);
return;
}
if (!existing.split(",").some((part) => part.trim().toLowerCase() === value.toLowerCase())) {
headers.set("Vary", `${existing}, ${value}`);
}
}
/** Returns the permitted value for Access-Control-Allow-Origin, or null when CORS is disabled or denied. */
export function resolveOrigin(request: Request, env: Env): string | null {
const origin = request.headers.get("Origin");
if (!origin) return null;
if (hasWildcardOrigin(env)) return "*";
return configuredOrigins(env).includes(origin) ? origin : null;
}
/** Builds an unauthenticated browser preflight response. */
export function preflightResponse(request: Request, env: Env): Response {
const headers = new Headers();
const origin = resolveOrigin(request, env);
if (origin) {
headers.set("Access-Control-Allow-Origin", origin);
headers.set("Access-Control-Allow-Methods", ALLOWED_METHODS);
headers.set("Access-Control-Allow-Headers", request.headers.get("Access-Control-Request-Headers") ?? DEFAULT_ALLOWED_HEADERS);
headers.set("Access-Control-Max-Age", "86400");
if (!hasWildcardOrigin(env)) appendVary(headers, "Origin");
}
return new Response(null, { status: 204, headers });
}
/** Adds CORS response headers without mutating the response supplied by the request handler. */
export function withCors(response: Response, request: Request, env: Env): Response {
const headers = new Headers(response.headers);
if (env.CORS_ALLOWED_ORIGINS?.trim() && !hasWildcardOrigin(env)) {
appendVary(headers, "Origin");
}
const origin = resolveOrigin(request, env);
if (origin) {
headers.set("Access-Control-Allow-Origin", origin);
headers.set("Access-Control-Expose-Headers", EXPOSED_HEADERS);
}
return new Response(response.body, { status: response.status, statusText: response.statusText, headers });
}
+36
View File
@@ -0,0 +1,36 @@
import openApiSpec from "../docs/openapi.yaml";
export const OPENAPI_PATH = "/openapi.yaml";
export const DOCS_PATH = "/docs";
export function openApiResponse(): Response {
return new Response(openApiSpec, {
headers: {
"Content-Type": "application/yaml; charset=utf-8",
"Cache-Control": "public, max-age=300",
},
});
}
export function docsResponse(): Response {
const html = `<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="color-scheme" content="dark light">
<title>I.R.I.S. S3 API reference</title>
</head>
<body>
<script id="api-reference" data-url="${OPENAPI_PATH}"></script>
<script src="https://cdn.jsdelivr.net/npm/@scalar/api-reference"></script>
</body>
</html>`;
return new Response(html, {
headers: {
"Content-Type": "text/html; charset=utf-8",
"Cache-Control": "public, max-age=300",
},
});
}
+2 -1
View File
@@ -220,9 +220,10 @@ export async function streamDownloadFromDrive(accessToken: string, bucket: strin
console.error(await downloadRes.text()); console.error(await downloadRes.text());
throw new Error("Download failed"); throw new Error("Download failed");
} }
if (!downloadRes.body) throw new Error("Download response had no body");
return { return {
body: downloadRes.body!, body: downloadRes.body,
contentType: file.mimeType || "application/octet-stream", contentType: file.mimeType || "application/octet-stream",
size: parseInt(file.size || "0", 10), size: parseInt(file.size || "0", 10),
id: file.id, id: file.id,
+15 -7
View File
@@ -1,5 +1,7 @@
import { verifySignature } from "./aws-signature"; import { verifySignature } from "./aws-signature";
import { isAllowedBucket, isPublicReadBucket } from "./bucket-access"; import { isAllowedBucket, isPublicReadBucket } from "./bucket-access";
import { preflightResponse, withCors } from "./cors";
import * as docs from "./docs";
import { getAccessToken } from "./google-drive"; import { getAccessToken } from "./google-drive";
import { MultipartUploadDO } from "./multipart-do"; import { MultipartUploadDO } from "./multipart-do";
import { dispatch } from "./router"; import { dispatch } from "./router";
@@ -10,27 +12,33 @@ export { MultipartUploadDO };
export default { export default {
async fetch(request: Request, env: Env, _ctx: ExecutionContext): Promise<Response> { async fetch(request: Request, env: Env, _ctx: ExecutionContext): Promise<Response> {
if (request.method === "OPTIONS") return new Response(null, { status: 204 }); if (request.method === "OPTIONS") return preflightResponse(request, env);
const url = new URL(request.url); const url = new URL(request.url);
if (env.ENABLE_DOCS !== "false") {
if (request.method === "GET" && url.pathname === docs.OPENAPI_PATH) return withCors(docs.openApiResponse(), request, env);
if (request.method === "GET" && url.pathname === docs.DOCS_PATH && !isAllowedBucket("docs", env)) return withCors(docs.docsResponse(), request, env);
}
const pathParts = url.pathname.split("/").filter(Boolean); const pathParts = url.pathname.split("/").filter(Boolean);
const bucket = pathParts[0] || ""; const bucket = pathParts[0] || "";
const objectKey = pathParts.slice(1).join("/"); const objectKey = pathParts.slice(1).join("/");
const resource = url.pathname || "/"; const resource = url.pathname || "/";
try { try {
if (!isAllowedBucket(bucket, env)) return s3Error("AccessDenied", 403, undefined, resource, request.method === "HEAD"); if (!isAllowedBucket(bucket, env)) return withCors(s3Error("AccessDenied", 403, undefined, resource, request.method === "HEAD"), request, env);
const isPublicRead = isPublicReadBucket(bucket, env) && (request.method === "GET" || request.method === "HEAD"); const isPublicRead = isPublicReadBucket(bucket, env) && (request.method === "GET" || request.method === "HEAD");
if (!isPublicRead && !(await verifySignature(request, env))) { const signature = isPublicRead ? { ok: true as const } : await verifySignature(request, env);
return s3Error("SignatureDoesNotMatch", 403, undefined, resource, request.method === "HEAD"); if (!signature.ok) {
return withCors(s3Error(signature.code, 403, signature.message, resource, request.method === "HEAD"), request, env);
} }
return await dispatch(request, env, await getAccessToken(env), bucket, objectKey); return withCors(await dispatch(request, env, await getAccessToken(env), bucket, objectKey), request, env);
} catch (error) { } catch (error) {
if (error instanceof S3Exception) return s3Error(error.code, error.status, error.message, resource, request.method === "HEAD", error.headers); if (error instanceof S3Exception) return withCors(s3Error(error.code, error.status, error.message, resource, request.method === "HEAD", error.headers), request, env);
console.error(JSON.stringify({ message: "request failed", error: error instanceof Error ? error.message : String(error), method: request.method, path: url.pathname })); console.error(JSON.stringify({ message: "request failed", error: error instanceof Error ? error.message : String(error), method: request.method, path: url.pathname }));
return s3Error("InternalError", 500, undefined, resource, request.method === "HEAD"); return withCors(s3Error("InternalError", 500, undefined, resource, request.method === "HEAD"), request, env);
} }
}, },
} satisfies ExportedHandler<Env>; } satisfies ExportedHandler<Env>;
+14 -8
View File
@@ -57,6 +57,12 @@ export class MultipartUploadDO extends DurableObject<Env> {
return this.ctx.storage.sql.exec<StateValueRow>("SELECT v FROM state WHERE k = ?", key).toArray()[0]?.v as T | undefined; return this.ctx.storage.sql.exec<StateValueRow>("SELECT v FROM state WHERE k = ?", key).toArray()[0]?.v as T | undefined;
} }
private requiredValue<T extends ArrayBuffer | string | number>(key: string): T {
const value = this.getValue<T>(key);
if (value === undefined) throw new Error(`Missing multipart upload state: ${key}`);
return value;
}
private setValue(key: string, value: ArrayBuffer | string | number | null): void { private setValue(key: string, value: ArrayBuffer | string | number | null): void {
this.ctx.storage.sql.exec("INSERT INTO state (k, v) VALUES (?, ?) ON CONFLICT(k) DO UPDATE SET v = excluded.v", key, value); this.ctx.storage.sql.exec("INSERT INTO state (k, v) VALUES (?, ?) ON CONFLICT(k) DO UPDATE SET v = excluded.v", key, value);
} }
@@ -97,7 +103,7 @@ export class MultipartUploadDO extends DurableObject<Env> {
private async resyncExpiredLease(inFlight: InFlight, requestId: string, partLen: number): Promise<BeginPartResult | null> { private async resyncExpiredLease(inFlight: InFlight, requestId: string, partLen: number): Promise<BeginPartResult | null> {
if (partLen !== inFlight.partLen) return { kind: "error", code: "InvalidPart", message: "A retried part must have the same decoded length" }; if (partLen !== inFlight.partLen) return { kind: "error", code: "InvalidPart", message: "A retried part must have the same decoded length" };
const accessToken = await getAccessToken(this.env); const accessToken = await getAccessToken(this.env);
const actualOffset = await queryStatus(this.getValue<string>("uploadUrl")!, accessToken); const actualOffset = await queryStatus(this.requiredValue<string>("uploadUrl"), accessToken);
const partStartFileOffset = inFlight.driveOffsetAtStart + inFlight.carryLen; const partStartFileOffset = inFlight.driveOffsetAtStart + inFlight.carryLen;
let skipBytes = 0; let skipBytes = 0;
if (actualOffset === inFlight.driveOffsetAtStart) { if (actualOffset === inFlight.driveOffsetAtStart) {
@@ -129,7 +135,7 @@ export class MultipartUploadDO extends DurableObject<Env> {
leaseExpiresAt: Date.now() + LEASE_MS, leaseExpiresAt: Date.now() + LEASE_MS,
}; };
this.setValue("inFlight", JSON.stringify(inFlight)); this.setValue("inFlight", JSON.stringify(inFlight));
return { kind: "admit", uploadUrl: this.getValue<string>("uploadUrl")!, driveOffset, carry, sendLen, skipBytes }; return { kind: "admit", uploadUrl: this.requiredValue<string>("uploadUrl"), driveOffset, carry, sendLen, skipBytes };
} }
async beginPart(requestId: string, partNumber: number, partLen: number): Promise<BeginPartResult> { async beginPart(requestId: string, partNumber: number, partLen: number): Promise<BeginPartResult> {
@@ -229,15 +235,15 @@ export class MultipartUploadDO extends DurableObject<Env> {
const accessToken = await getAccessToken(this.env); const accessToken = await getAccessToken(this.env);
let metadata: DriveUploadResult; let metadata: DriveUploadResult;
if (total === 0) { if (total === 0) {
await cancelSession(this.getValue<string>("uploadUrl")!, accessToken); await cancelSession(this.requiredValue<string>("uploadUrl"), accessToken);
metadata = await createEmptyFile(accessToken, { metadata = await createEmptyFile(accessToken, {
name: this.getValue<string>("fileName")!, name: this.requiredValue<string>("fileName"),
parents: [this.getValue<string>("parentFolderId")!], parents: [this.requiredValue<string>("parentFolderId")],
mimeType: this.getValue<string>("mimeType")!, mimeType: this.requiredValue<string>("mimeType"),
existingFileId: this.getValue<string>("existingFileId"), existingFileId: this.getValue<string>("existingFileId"),
}); });
} else { } else {
metadata = await putFinalChunk(this.getValue<string>("uploadUrl")!, accessToken, driveOffset, total, carry); metadata = await putFinalChunk(this.requiredValue<string>("uploadUrl"), accessToken, driveOffset, total, carry);
} }
const partEtags = stored.map((part) => part.etag); const partEtags = stored.map((part) => part.etag);
await this.ctx.storage.deleteAlarm(); await this.ctx.storage.deleteAlarm();
@@ -259,7 +265,7 @@ export class MultipartUploadDO extends DurableObject<Env> {
async abort(): Promise<boolean> { async abort(): Promise<boolean> {
if (!this.hasUpload()) return false; if (!this.hasUpload()) return false;
const accessToken = await getAccessToken(this.env); const accessToken = await getAccessToken(this.env);
await cancelSession(this.getValue<string>("uploadUrl")!, accessToken); await cancelSession(this.requiredValue<string>("uploadUrl"), accessToken);
await this.ctx.storage.deleteAlarm(); await this.ctx.storage.deleteAlarm();
await this.ctx.storage.deleteAll(); await this.ctx.storage.deleteAll();
for (const waiter of this.waiters.values()) { for (const waiter of this.waiters.values()) {
+2 -1
View File
@@ -1,10 +1,11 @@
import { escapeXml } from "./s3-xml"; import { escapeXml } from "./s3-xml";
export type S3ErrorCode = "AccessDenied" | "SignatureDoesNotMatch" | "NoSuchKey" | "NoSuchUpload" | "InvalidPart" | "InvalidPartOrder" | "EntityTooLarge" | "MalformedXML" | "InvalidArgument" | "MethodNotAllowed" | "NotImplemented" | "SlowDown" | "InternalError"; export type S3ErrorCode = "AccessDenied" | "SignatureDoesNotMatch" | "RequestTimeTooSkewed" | "NoSuchKey" | "NoSuchUpload" | "InvalidPart" | "InvalidPartOrder" | "EntityTooLarge" | "MalformedXML" | "InvalidArgument" | "MethodNotAllowed" | "NotImplemented" | "SlowDown" | "InternalError";
const DEFAULT_MESSAGES: Record<S3ErrorCode, string> = { const DEFAULT_MESSAGES: Record<S3ErrorCode, string> = {
AccessDenied: "Access Denied", AccessDenied: "Access Denied",
SignatureDoesNotMatch: "The request signature we calculated does not match the signature you provided.", SignatureDoesNotMatch: "The request signature we calculated does not match the signature you provided.",
RequestTimeTooSkewed: "The difference between the request time and the server's time is too large.",
NoSuchKey: "The specified key does not exist.", NoSuchKey: "The specified key does not exist.",
NoSuchUpload: "The specified multipart upload does not exist.", NoSuchUpload: "The specified multipart upload does not exist.",
InvalidPart: "One or more of the specified parts could not be found.", InvalidPart: "One or more of the specified parts could not be found.",
+4
View File
@@ -0,0 +1,4 @@
declare module "*.yaml" {
const content: string;
export default content;
}
+2
View File
@@ -12,6 +12,8 @@ export interface Env {
PUBLIC_READ_BUCKETS?: string; PUBLIC_READ_BUCKETS?: string;
ALLOW_MULTIPART?: string; ALLOW_MULTIPART?: string;
ETAG_STYLE?: "md5" | "multipart"; ETAG_STYLE?: "md5" | "multipart";
CORS_ALLOWED_ORIGINS?: string;
ENABLE_DOCS?: string;
} }
export interface GoogleDriveFile { export interface GoogleDriveFile {
+119
View File
@@ -0,0 +1,119 @@
import { AwsClient } from "aws4fetch";
import { afterEach, describe, expect, it, vi } from "vitest";
import { preflightResponse, withCors } from "../src/cors";
import worker from "../src/index";
import type { Env } from "../src/types";
import { env } from "cloudflare:test";
const ENV = env as unknown as Env;
const ENDPOINT = "https://s3-api.example.com";
const ORIGIN = "http://localhost:5173";
const CTX = { waitUntil: vi.fn(), passThroughOnException: vi.fn() } as unknown as ExecutionContext;
async function signed(path: string, init: RequestInit): Promise<Request> {
const aws = new AwsClient({ accessKeyId: ENV.ACCESS_KEY, secretAccessKey: ENV.SECRET_KEY, region: ENV.REGION, service: "s3" });
const bodyLength = typeof init.body === "string" ? new TextEncoder().encode(init.body).byteLength : init.body instanceof Uint8Array ? init.body.byteLength : undefined;
return aws.sign(`${ENDPOINT}${path}`, {
...init,
headers: { "x-amz-content-sha256": "UNSIGNED-PAYLOAD", ...(bodyLength === undefined ? {} : { "x-amz-decoded-content-length": String(bodyLength) }), ...init.headers },
});
}
function fakeGoogleFetch(input: string | URL | Request, init?: RequestInit): Promise<Response> {
const request = input instanceof Request ? input : new Request(input, init);
const url = new URL(request.url);
if (url.hostname === "oauth2.googleapis.com") return Promise.resolve(Response.json({ access_token: "token", expires_in: 3600 }));
if (url.pathname === "/drive/v3/files" && request.method === "GET") return Promise.resolve(Response.json({ files: [] }));
if (url.pathname === "/drive/v3/files" && request.method === "POST") return Promise.resolve(Response.json({ id: "folder-1" }));
if (url.pathname.startsWith("/upload/drive/v3/files")) return Promise.resolve(new Response(null, { headers: { Location: "https://www.googleapis.com/upload/session/test" } }));
if (url.pathname === "/upload/session/test") return Promise.resolve(Response.json({ id: "file-1", name: "file.txt", md5Checksum: "d41d8cd98f00b204e9800998ecf8427e" }));
return Promise.resolve(new Response("Not Found", { status: 404 }));
}
afterEach(() => vi.unstubAllGlobals());
describe("CORS", () => {
it("returns allow-list preflight headers", async () => {
const response = preflightResponse(
new Request(`${ENDPOINT}/test-bucket/file.txt`, {
method: "OPTIONS",
headers: { Origin: ORIGIN, "Access-Control-Request-Method": "PUT", "Access-Control-Request-Headers": "content-type,x-amz-date" },
}),
ENV,
);
expect(response.status).toBe(204);
expect(response.headers.get("Access-Control-Allow-Origin")).toBe(ORIGIN);
expect(response.headers.get("Access-Control-Allow-Methods")).toBe("GET, HEAD, PUT, POST, DELETE, OPTIONS");
expect(response.headers.get("Access-Control-Allow-Headers")).toBe("content-type,x-amz-date");
expect(response.headers.get("Access-Control-Max-Age")).toBe("86400");
expect(response.headers.get("Vary")).toBe("Origin");
});
it("returns a bare 204 for a disallowed origin", async () => {
const response = preflightResponse(new Request(`${ENDPOINT}/test-bucket/file.txt`, { method: "OPTIONS", headers: { Origin: "https://not-allowed.example", "Access-Control-Request-Method": "PUT" } }), ENV);
expect(response.status).toBe(204);
expect([...response.headers]).toEqual([]);
});
it("keeps OPTIONS requests without an Origin byte-compatible with the prior bare 204", () => {
const response = preflightResponse(new Request(`${ENDPOINT}/test-bucket/file.txt`, { method: "OPTIONS" }), ENV);
expect(response.status).toBe(204);
expect([...response.headers]).toEqual([]);
});
it("exposes ETag for successful PUT responses", async () => {
vi.stubGlobal("fetch", vi.fn(fakeGoogleFetch));
const request = await signed("/test-bucket/file.txt", { method: "PUT", body: "hello", headers: { Origin: ORIGIN, "Content-Type": "text/plain" } });
const response = await worker.fetch(request, ENV, CTX);
expect(response.status).toBe(200);
expect(response.headers.get("ETag")).toBe('"d41d8cd98f00b204e9800998ecf8427e"');
expect(response.headers.get("Access-Control-Allow-Origin")).toBe(ORIGIN);
expect(response.headers.get("Access-Control-Expose-Headers")).toContain("ETag");
});
it("adds CORS headers to access-denied and missing-key errors", async () => {
const denied = await worker.fetch(new Request(`${ENDPOINT}/not-a-bucket`, { headers: { Origin: ORIGIN } }), ENV, CTX);
expect(denied.status).toBe(403);
expect(denied.headers.get("Access-Control-Allow-Origin")).toBe(ORIGIN);
vi.stubGlobal("fetch", vi.fn(fakeGoogleFetch));
const missing = await worker.fetch(await signed("/test-bucket/missing.txt", { method: "GET", headers: { Origin: ORIGIN } }), ENV, CTX);
expect(missing.status).toBe(404);
expect(await missing.text()).toContain("<Code>NoSuchKey</Code>");
expect(missing.headers.get("Access-Control-Allow-Origin")).toBe(ORIGIN);
});
it("does not alter the S3 response for a disallowed Origin", async () => {
const response = await worker.fetch(new Request(`${ENDPOINT}/not-a-bucket`, { headers: { Origin: "https://not-allowed.example" } }), ENV, CTX);
expect(response.status).toBe(403);
expect(await response.text()).toContain("<Code>AccessDenied</Code>");
expect(response.headers.get("Access-Control-Allow-Origin")).toBeNull();
expect(response.headers.get("Access-Control-Expose-Headers")).toBeNull();
expect(response.headers.get("Vary")).toBe("Origin");
});
it("does not attach Access-Control headers to server-to-server responses without Origin", () => {
const response = withCors(new Response("S3 response", { status: 200, headers: { ETag: '"etag"' } }), new Request(`${ENDPOINT}/test-bucket/file.txt`), ENV);
expect(response.status).toBe(200);
expect(response.headers.get("ETag")).toBe('"etag"');
expect(response.headers.get("Access-Control-Allow-Origin")).toBeNull();
expect(response.headers.get("Access-Control-Expose-Headers")).toBeNull();
expect(response.headers.get("Vary")).toBe("Origin");
});
it("emits no CORS headers when CORS_ALLOWED_ORIGINS is unset", () => {
const request = new Request(`${ENDPOINT}/test-bucket/file.txt`, { headers: { Origin: ORIGIN } });
const response = withCors(new Response(null, { headers: { ETag: '"etag"' } }), request, { ...ENV, CORS_ALLOWED_ORIGINS: undefined });
expect(response.headers.get("Access-Control-Allow-Origin")).toBeNull();
expect(response.headers.get("Access-Control-Expose-Headers")).toBeNull();
expect(response.headers.get("Vary")).toBeNull();
});
});
+39
View File
@@ -0,0 +1,39 @@
import { describe, expect, it, vi } from "vitest";
import worker from "../src/index";
import type { Env } from "../src/types";
import { env } from "cloudflare:test";
const ENV = env as unknown as Env;
const ENDPOINT = "https://s3-api.example.com";
const CTX = { waitUntil: vi.fn(), passThroughOnException: vi.fn() } as unknown as ExecutionContext;
describe("API documentation routes", () => {
it("serves the Scalar shell and raw OpenAPI document", async () => {
const docs = await worker.fetch(new Request(`${ENDPOINT}/docs`), ENV, CTX);
expect(docs.status).toBe(200);
expect(docs.headers.get("Content-Type")).toContain("text/html");
expect(await docs.text()).toContain("@scalar/api-reference");
const spec = await worker.fetch(new Request(`${ENDPOINT}/openapi.yaml`), ENV, CTX);
expect(spec.status).toBe(200);
expect(spec.headers.get("Content-Type")).toContain("application/yaml");
expect(new TextDecoder().decode(await spec.arrayBuffer())).toContain("openapi: 3.1.0");
});
it("bypasses documentation routes when disabled", async () => {
const disabled = { ...ENV, ENABLE_DOCS: "false" };
for (const path of ["/docs", "/openapi.yaml"]) {
const response = await worker.fetch(new Request(`${ENDPOINT}${path}`), disabled, CTX);
expect(response.status).toBe(403);
}
});
it("does not claim /docs when docs is a configured bucket", async () => {
const withDocsBucket = { ...ENV, ALLOWED_BUCKETS: "test-bucket,docs" };
const response = await worker.fetch(new Request(`${ENDPOINT}/docs`), withDocsBucket, CTX);
expect(response.status).toBe(403);
expect(await response.text()).toContain("<Code>SignatureDoesNotMatch</Code>");
});
});
+64 -9
View File
@@ -56,7 +56,9 @@ class FakeDrive {
if (url.pathname === "/drive/v3/files" && request.method === "POST") return this.createMetadata(await request.json<Record<string, unknown>>()); if (url.pathname === "/drive/v3/files" && request.method === "POST") return this.createMetadata(await request.json<Record<string, unknown>>());
if (url.pathname.startsWith("/drive/v3/files/") && url.searchParams.get("alt") === "media") return this.download(url, request); if (url.pathname.startsWith("/drive/v3/files/") && url.searchParams.get("alt") === "media") return this.download(url, request);
if (url.pathname.startsWith("/drive/v3/files/") && request.method === "DELETE") { if (url.pathname.startsWith("/drive/v3/files/") && request.method === "DELETE") {
this.files.delete(url.pathname.split("/").at(-1)!); const fileId = url.pathname.split("/").at(-1);
if (!fileId) return new Response("Not Found", { status: 404 });
this.files.delete(fileId);
return new Response(null, { status: 204 }); return new Response(null, { status: 204 });
} }
if (url.pathname.startsWith("/upload/drive/v3/files") && url.searchParams.get("uploadType") === "resumable") return this.initialize(url, request); if (url.pathname.startsWith("/upload/drive/v3/files") && url.searchParams.get("uploadType") === "resumable") return this.initialize(url, request);
@@ -121,7 +123,8 @@ class FakeDrive {
} }
private async upload(url: URL, request: Request): Promise<Response> { private async upload(url: URL, request: Request): Promise<Response> {
const id = url.pathname.split("/").at(-1)!; const id = url.pathname.split("/").at(-1);
if (!id) return new Response(null, { status: 404 });
const session = this.sessions.get(id); const session = this.sessions.get(id);
if (!session) return new Response(null, { status: 404 }); if (!session) return new Response(null, { status: 404 });
if (request.method === "DELETE") { if (request.method === "DELETE") {
@@ -159,7 +162,9 @@ class FakeDrive {
} }
private download(url: URL, request: Request): Response { private download(url: URL, request: Request): Response {
const file = this.files.get(url.pathname.split("/").at(-1)!); const fileId = url.pathname.split("/").at(-1);
if (!fileId) return new Response(null, { status: 404 });
const file = this.files.get(fileId);
if (!file) return new Response(null, { status: 404 }); if (!file) return new Response(null, { status: 404 });
const range = request.headers.get("Range"); const range = request.headers.get("Range");
if (!range) return new Response(file.data, { headers: { "Content-Length": String(file.data.byteLength) } }); if (!range) return new Response(file.data, { headers: { "Content-Length": String(file.data.byteLength) } });
@@ -204,6 +209,18 @@ async function signed(path: string, init: RequestInit): Promise<Request> {
}); });
} }
async function presigned(path: string, init: RequestInit, options: { datetime?: string; accessKeyId?: string; expires?: string } = {}): Promise<Request> {
const url = new URL(`${ENDPOINT}${path}`);
url.searchParams.set("X-Amz-Expires", options.expires ?? "60");
const aws = new AwsClient({ accessKeyId: options.accessKeyId ?? ENV.ACCESS_KEY, secretAccessKey: ENV.SECRET_KEY, region: ENV.REGION, service: "s3" });
const bodyLength = typeof init.body === "string" ? new TextEncoder().encode(init.body).byteLength : init.body instanceof Uint8Array ? init.body.byteLength : undefined;
return aws.sign(url.toString(), {
...init,
headers: { "x-amz-content-sha256": "UNSIGNED-PAYLOAD", ...(bodyLength === undefined ? {} : { "x-amz-decoded-content-length": String(bodyLength) }), ...init.headers },
aws: { signQuery: true, ...(options.datetime ? { datetime: options.datetime } : {}) },
});
}
let drive: FakeDrive; let drive: FakeDrive;
beforeEach(async () => { beforeEach(async () => {
@@ -226,7 +243,10 @@ describe("S3 compatibility", () => {
const second = await worker.fetch(await signed("/test-bucket/file.txt", { method: "PUT", body: "second" }), ENV, CTX); const second = await worker.fetch(await signed("/test-bucket/file.txt", { method: "PUT", body: "second" }), ENV, CTX);
expect(second.status).toBe(200); expect(second.status).toBe(200);
expect([...drive.files.values()].filter((file) => file.name === "file.txt")).toHaveLength(1); expect([...drive.files.values()].filter((file) => file.name === "file.txt")).toHaveLength(1);
expect(new TextDecoder().decode([...drive.files.values()].find((file) => file.name === "file.txt")!.data)).toBe("second"); const stored = [...drive.files.values()].find((file) => file.name === "file.txt");
expect(stored).toBeDefined();
if (!stored) throw new Error("Overwritten object was not stored");
expect(new TextDecoder().decode(stored.data)).toBe("second");
}); });
it("sets Last-Modified on GET and HEAD from Drive's modifiedTime", async () => { it("sets Last-Modified on GET and HEAD from Drive's modifiedTime", async () => {
@@ -264,6 +284,33 @@ describe("S3 compatibility", () => {
expect(response.status).toBe(200); expect(response.status).toBe(200);
}); });
it("rejects expired presigned URLs and accepts unexpired ones", async () => {
const expired = await worker.fetch(await presigned("/test-bucket/file.txt", { method: "GET" }, { datetime: "20200101T000000Z", expires: "60" }), ENV, CTX);
expect(expired.status).toBe(403);
expect(await expired.text()).toContain("<Code>AccessDenied</Code>");
});
it("accepts an unexpired presigned URL", async () => {
await worker.fetch(await signed("/test-bucket/presigned.txt", { method: "PUT", body: "hello" }), ENV, CTX);
const response = await worker.fetch(await presigned("/test-bucket/presigned.txt", { method: "GET" }), ENV, CTX);
expect(response.status).toBe(200);
expect(await response.text()).toBe("hello");
});
it("rejects a Credential access key that does not match ACCESS_KEY", async () => {
const response = await worker.fetch(await presigned("/test-bucket/file.txt", { method: "GET" }, { accessKeyId: "unexpected-access-key" }), ENV, CTX);
expect(response.status).toBe(403);
expect(await response.text()).toContain("<Code>AccessDenied</Code>");
});
it("rejects header-authenticated requests outside the 15-minute clock skew", async () => {
const aws = new AwsClient({ accessKeyId: ENV.ACCESS_KEY, secretAccessKey: ENV.SECRET_KEY, region: ENV.REGION, service: "s3" });
const request = await aws.sign(`${ENDPOINT}/test-bucket/file.txt`, { method: "GET", aws: { datetime: "20200101T000000Z" } });
const response = await worker.fetch(request, ENV, CTX);
expect(response.status).toBe(403);
expect(await response.text()).toContain("<Code>RequestTimeTooSkewed</Code>");
});
it("decodes both aws-chunked framing variants across arbitrary boundaries", async () => { it("decodes both aws-chunked framing variants across arbitrary boundaries", async () => {
const payload = bytes(70_013); const payload = bytes(70_013);
for (const trailer of [true, false]) { for (const trailer of [true, false]) {
@@ -282,20 +329,28 @@ describe("S3 compatibility", () => {
const source = bytes(1_500_123); const source = bytes(1_500_123);
const create = await worker.fetch(await signed("/test-bucket/big.bin?uploads", { method: "POST", headers: { "Content-Type": "application/octet-stream" } }), ENV, CTX); const create = await worker.fetch(await signed("/test-bucket/big.bin?uploads", { method: "POST", headers: { "Content-Type": "application/octet-stream" } }), ENV, CTX);
expect(create.status).toBe(200); expect(create.status).toBe(200);
const uploadId = /<UploadId>([^<]+)<\/UploadId>/.exec(await create.text())![1]; const uploadId = /<UploadId>([^<]+)<\/UploadId>/.exec(await create.text())?.[1];
expect(uploadId).toBeDefined();
if (!uploadId) throw new Error("Multipart initiation did not return an upload ID");
const completed: Array<{ partNumber: number; etag: string }> = []; const completed: Array<{ partNumber: number; etag: string }> = [];
for (let index = 0, offset = 0; offset < source.byteLength; index++) { for (let index = 0, offset = 0; offset < source.byteLength; index++) {
const end = Math.min(source.byteLength, offset + 500_000); const end = Math.min(source.byteLength, offset + 500_000);
const part = await worker.fetch(await signed(`/test-bucket/big.bin?partNumber=${index + 1}&uploadId=${encodeURIComponent(uploadId)}`, { method: "PUT", body: source.slice(offset, end) }), ENV, CTX); const part = await worker.fetch(await signed(`/test-bucket/big.bin?partNumber=${index + 1}&uploadId=${encodeURIComponent(uploadId)}`, { method: "PUT", body: source.slice(offset, end) }), ENV, CTX);
expect(part.status).toBe(200); expect(part.status).toBe(200);
completed.push({ partNumber: index + 1, etag: part.headers.get("ETag")!.replaceAll('"', "") }); const partEtag = part.headers.get("ETag");
expect(partEtag).toBeDefined();
if (!partEtag) throw new Error(`Multipart part ${index + 1} did not return an ETag`);
completed.push({ partNumber: index + 1, etag: partEtag.replaceAll('"', "") });
offset = end; offset = end;
} }
const xml = `<CompleteMultipartUpload>${completed.map((part) => `<Part><PartNumber>${part.partNumber}</PartNumber><ETag>"${part.etag}"</ETag></Part>`).join("")}</CompleteMultipartUpload>`; const xml = `<CompleteMultipartUpload>${completed.map((part) => `<Part><PartNumber>${part.partNumber}</PartNumber><ETag>"${part.etag}"</ETag></Part>`).join("")}</CompleteMultipartUpload>`;
const result = await worker.fetch(await signed(`/test-bucket/big.bin?uploadId=${encodeURIComponent(uploadId)}`, { method: "POST", body: xml }), ENV, CTX); const result = await worker.fetch(await signed(`/test-bucket/big.bin?uploadId=${encodeURIComponent(uploadId)}`, { method: "POST", body: xml }), ENV, CTX);
expect(result.status).toBe(200); expect(result.status).toBe(200);
expect(await result.text()).toMatch(/<ETag>"[0-9a-f]{32}"<\/ETag>/); expect(await result.text()).toMatch(/<ETag>"[0-9a-f]{32}"<\/ETag>/);
const stored = [...drive.files.values()].find((file) => file.name === "big.bin")!.data; const storedFile = [...drive.files.values()].find((file) => file.name === "big.bin");
expect(storedFile).toBeDefined();
if (!storedFile) throw new Error("Completed multipart object was not stored");
const stored = storedFile.data;
expect(stored.byteLength).toBe(source.byteLength); expect(stored.byteLength).toBe(source.byteLength);
expect(fakeMd5(stored)).toBe(fakeMd5(source)); expect(fakeMd5(stored)).toBe(fakeMd5(source));
}); });
@@ -317,13 +372,13 @@ describe("S3 compatibility", () => {
CTX, CTX,
); );
expect(response.status).toBe(200); expect(response.status).toBe(200);
expect([...drive.files.values()].find((file) => file.name === "chunked.bin")!.data).toEqual(source); expect([...drive.files.values()].find((file) => file.name === "chunked.bin")?.data).toEqual(source);
}); });
it("supports an empty PutObject", async () => { it("supports an empty PutObject", async () => {
const response = await worker.fetch(await signed("/test-bucket/empty", { method: "PUT", body: new Uint8Array() }), ENV, CTX); const response = await worker.fetch(await signed("/test-bucket/empty", { method: "PUT", body: new Uint8Array() }), ENV, CTX);
expect(response.status).toBe(200); expect(response.status).toBe(200);
expect([...drive.files.values()].find((file) => file.name === "empty")!.data.byteLength).toBe(0); expect([...drive.files.values()].find((file) => file.name === "empty")?.data.byteLength).toBe(0);
}); });
it("lists nested keys under a prefix, as CommonPrefixes with a delimiter and recursively without one", async () => { it("lists nested keys under a prefix, as CommonPrefixes with a delimiter and recursively without one", async () => {
+2
View File
@@ -16,6 +16,8 @@ export default defineConfig({
ALLOWED_BUCKETS: "test-bucket,empty-bucket,my-bucket", ALLOWED_BUCKETS: "test-bucket,empty-bucket,my-bucket",
ALLOW_MULTIPART: "true", ALLOW_MULTIPART: "true",
ETAG_STYLE: "md5", ETAG_STYLE: "md5",
CORS_ALLOWED_ORIGINS: "http://localhost:5173",
ENABLE_DOCS: "true",
}, },
}, },
}), }),
+2272 -2750
View File
File diff suppressed because it is too large Load Diff
+52 -46
View File
@@ -3,69 +3,75 @@
* https://developers.cloudflare.com/workers/wrangler/configuration/ * https://developers.cloudflare.com/workers/wrangler/configuration/
*/ */
{ {
"$schema": "node_modules/wrangler/config-schema.json", "$schema": "node_modules/wrangler/config-schema.json",
"name": "iris", "name": "iris",
"main": "src/index.ts", "main": "src/index.ts",
"compatibility_date": "2025-09-27", "compatibility_date": "2025-09-27",
"vars": { "vars": {
"ALLOW_MULTIPART": "true", "ALLOW_MULTIPART": "true",
"ETAG_STYLE": "md5" "ETAG_STYLE": "md5",
}, "CORS_ALLOWED_ORIGINS": "",
"observability": { "ENABLE_DOCS": "true"
"enabled": true },
}, "rules": [
"durable_objects": { {
"bindings": [ "type": "Text",
{ "globs": ["docs/openapi.yaml"]
"name": "MPU", }
"class_name": "MultipartUploadDO" ],
} "observability": {
] "enabled": true
}, },
"migrations": [ "durable_objects": {
{ "bindings": [
"tag": "v1", {
"new_sqlite_classes": [ "name": "MPU",
"MultipartUploadDO" "class_name": "MultipartUploadDO"
] }
} ]
], },
"kv_namespaces": [ "migrations": [
{ {
"binding": "AUTH_KV", "tag": "v1",
"id": "2113be46ce514e088572d0ef7459c1bf" "new_sqlite_classes": ["MultipartUploadDO"]
}, }
{ ],
"binding": "FOLDER_CACHE", "kv_namespaces": [
"id": "6d2f9904a3ff4471b0f04025c1ae87cd" {
} "binding": "AUTH_KV",
] "id": "2113be46ce514e088572d0ef7459c1bf"
/** },
{
"binding": "FOLDER_CACHE",
"id": "6d2f9904a3ff4471b0f04025c1ae87cd"
}
]
/**
* Smart Placement * Smart Placement
* https://developers.cloudflare.com/workers/configuration/smart-placement/#smart-placement * https://developers.cloudflare.com/workers/configuration/smart-placement/#smart-placement
*/ */
// "placement": { "mode": "smart" } // "placement": { "mode": "smart" }
/** /**
* Bindings * Bindings
* Bindings allow your Worker to interact with resources on the Cloudflare Developer Platform, including * Bindings allow your Worker to interact with resources on the Cloudflare Developer Platform, including
* databases, object storage, AI inference, real-time communication and more. * databases, object storage, AI inference, real-time communication and more.
* https://developers.cloudflare.com/workers/runtime-apis/bindings/ * https://developers.cloudflare.com/workers/runtime-apis/bindings/
*/ */
/** /**
* Environment Variables * Environment Variables
* https://developers.cloudflare.com/workers/wrangler/configuration/#environment-variables * https://developers.cloudflare.com/workers/wrangler/configuration/#environment-variables
* Note: Use secrets to store sensitive data. * Note: Use secrets to store sensitive data.
* https://developers.cloudflare.com/workers/configuration/secrets/ * https://developers.cloudflare.com/workers/configuration/secrets/
*/ */
// "vars": { "MY_VARIABLE": "production_value" } // "vars": { "MY_VARIABLE": "production_value" }
/** /**
* Static Assets * Static Assets
* https://developers.cloudflare.com/workers/static-assets/binding/ * https://developers.cloudflare.com/workers/static-assets/binding/
*/ */
// "assets": { "directory": "./public/", "binding": "ASSETS" } // "assets": { "directory": "./public/", "binding": "ASSETS" }
/** /**
* Service Bindings (communicate between multiple Workers) * Service Bindings (communicate between multiple Workers)
* https://developers.cloudflare.com/workers/wrangler/configuration/#service-bindings * https://developers.cloudflare.com/workers/wrangler/configuration/#service-bindings
*/ */
// "services": [ { "binding": "MY_SERVICE", "service": "my-service" } ] // "services": [ { "binding": "MY_SERVICE", "service": "my-service" } ]
} }