mirror of
https://github.com/Nezumi-2711/google-drive-s3.git
synced 2026-09-22 05:31:47 +00:00
feat: add the openapi docs
This commit is contained in:
@@ -0,0 +1,27 @@
|
||||
# Local-only Cloudflare Worker bindings. Copy to .dev.vars and replace each value.
|
||||
# Secrets must never be committed.
|
||||
# AUTH_KV, FOLDER_CACHE, and MPU are configured bindings in wrangler.jsonc, not .dev.vars values.
|
||||
|
||||
# S3 Signature V4 credentials held by CLI clients or your BFF.
|
||||
ACCESS_KEY=replace-with-access-key-id
|
||||
SECRET_KEY=replace-with-long-random-secret
|
||||
REGION=auto
|
||||
|
||||
# Google OAuth credentials with Drive API access.
|
||||
GOOGLE_CLIENT_ID=replace-with-google-oauth-client-id
|
||||
GOOGLE_CLIENT_SECRET=replace-with-google-oauth-client-secret
|
||||
GOOGLE_REFRESH_TOKEN=replace-with-google-refresh-token
|
||||
|
||||
# Comma-separated, exact bucket names. Unset denies every bucket.
|
||||
ALLOWED_BUCKETS=assets
|
||||
# Optional comma-separated subset of ALLOWED_BUCKETS that permits unsigned GET/HEAD.
|
||||
PUBLIC_READ_BUCKETS=
|
||||
|
||||
# Durable Object multipart uploads and ETag result behavior.
|
||||
ALLOW_MULTIPART=true
|
||||
ETAG_STYLE=md5
|
||||
|
||||
# Browser CORS: comma-separated exact origins or *. Leave empty to disable CORS.
|
||||
CORS_ALLOWED_ORIGINS=http://localhost:5173
|
||||
# Set false to disable /docs and /openapi.yaml.
|
||||
ENABLE_DOCS=true
|
||||
@@ -3,6 +3,10 @@ I.R.I.S. (Integrated Reliable Interop Storage)
|
||||
|
||||
Use Cloudflare Workers to turn your Google Drive into S3 object storage at no extra cost.
|
||||
|
||||
## API documentation
|
||||
|
||||
With `ENABLE_DOCS` left enabled (the default), each deployment serves an interactive API reference at `/docs` and the raw OpenAPI 3.1 document at `/openapi.yaml`. Frontend-specific setup, authentication details, limitations, and runnable examples live in [`docs/`](./docs/README.md).
|
||||
|
||||
## About
|
||||
This is a Workers script that converts the Google Drive API into an S3-compatible API. Turn your Google Drive into object storage at no extra cost!
|
||||
|
||||
@@ -74,6 +78,8 @@ https://developers.cloudflare.com/workers/configuration/secrets/#via-the-dashboa
|
||||
| `GOOGLE_CLIENT_ID`, `GOOGLE_CLIENT_SECRET`, `GOOGLE_REFRESH_TOKEN` | Google API credentials obtained from rclone. |
|
||||
| `ALLOWED_BUCKETS` | Set the buckets allowed, separated by `,`. A directory with the bucket name will be created directly under Google Drive. |
|
||||
| `PUBLIC_READ_BUCKETS` | *(Optional)* Buckets that allow unauthenticated GET/HEAD access without signature, separated by `,`. Write operations (PUT/POST/DELETE) still require authentication. Must be a subset of `ALLOWED_BUCKETS`. |
|
||||
| `CORS_ALLOWED_ORIGINS` | *(Optional)* Comma-separated exact browser origins, or `*`. Unset emits no CORS headers. |
|
||||
| `ENABLE_DOCS` | *(Optional)* Set to `false` to disable `/docs` and `/openapi.yaml`; enabled by default. |
|
||||
|
||||
### 4. Enable Multipart Uploads
|
||||
|
||||
@@ -95,5 +101,12 @@ Google Drive's free tier has 15 GB total storage, and Google applies a 750 GB da
|
||||
|
||||
|
||||
### 5. CORS Configuration
|
||||
If you need to configure CORS, set up your own domain for Workers and use Cloudflare's Response Header Transform Rules to add the necessary headers.
|
||||
https://developers.cloudflare.com/rules/transform/response-header-modification/
|
||||
I.R.I.S. provides native, deny-by-default CORS handling. Set `CORS_ALLOWED_ORIGINS` to a comma-separated list of exact origins:
|
||||
|
||||
```ini
|
||||
CORS_ALLOWED_ORIGINS=https://app.example.com,http://localhost:5173
|
||||
```
|
||||
|
||||
Use `*` only for a public, credential-free integration. Leave the value unset to emit no CORS headers, preserving CLI-only behavior. Allowed origins receive preflight support for S3 methods and the browser-readable response headers `ETag`, `Content-Range`, `Content-Length`, `Last-Modified`, `Accept-Ranges`, and `x-amz-request-id`. Server-to-server clients do not need CORS: requests without an `Origin` retain their normal S3 response and receive no `Access-Control-*` headers. Exact origin allow-lists add the harmless `Vary: Origin` response header for cache correctness.
|
||||
|
||||
For browser uploads, keep `SECRET_KEY` in a BFF and give the browser short-lived presigned URLs. See the [frontend integration guide](./docs/integration-guide.md).
|
||||
|
||||
+1
-1
@@ -7,7 +7,7 @@
|
||||
},
|
||||
"files": {
|
||||
"ignoreUnknown": true,
|
||||
"includes": ["**", "!node_modules", "!.next", "!dist", "!build", "!src/app/globals.css"]
|
||||
"includes": ["**", "!node_modules", "!.next", "!dist", "!build", "!src/app/globals.css", "!worker-configuration.d.ts"]
|
||||
},
|
||||
"formatter": {
|
||||
"lineWidth": 320,
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
# I.R.I.S. frontend integration docs
|
||||
|
||||
I.R.I.S. exposes a focused S3-compatible API on top of Google Drive. The live API reference is available from every enabled Worker deployment at [`/docs`](/docs), with the source OpenAPI document at [`/openapi.yaml`](/openapi.yaml).
|
||||
|
||||
## Start here
|
||||
|
||||
1. Configure the Worker with `CORS_ALLOWED_ORIGINS` for every browser origin that will upload or download objects.
|
||||
2. Keep `SECRET_KEY` in a backend-for-frontend (BFF), not in browser code.
|
||||
3. Let the BFF issue short-lived presigned URLs for a constrained bucket/key/method.
|
||||
4. Upload or download through the presigned URL from the browser.
|
||||
|
||||
| Document | Purpose |
|
||||
| --- | --- |
|
||||
| [Integration guide](./integration-guide.md) | Browser upload, multipart upload, listing, CORS, and recommended architecture. |
|
||||
| [Authentication](./authentication.md) | The precise AWS Signature V4 behavior enforced by I.R.I.S. |
|
||||
| [Limitations](./limitations.md) | Unsupported S3 features and Google Drive operational limits. |
|
||||
| [`examples/bff-presign.ts`](./examples/bff-presign.ts) | A Workers BFF endpoint that creates short-lived PUT URLs. |
|
||||
| [`examples/browser-upload.ts`](./examples/browser-upload.ts) | Single object upload from a browser. |
|
||||
| [`examples/browser-multipart.ts`](./examples/browser-multipart.ts) | Strictly sequential browser multipart upload. |
|
||||
|
||||
## Browser security model
|
||||
|
||||
```text
|
||||
Browser ── POST /presign ──> BFF (holds SECRET_KEY)
|
||||
│ │
|
||||
└──── presigned S3 request ────┴──> I.R.I.S. Worker ──> Google Drive
|
||||
```
|
||||
|
||||
The browser never receives `SECRET_KEY`. Presigned URLs must be short lived and scoped to the single object operation the browser needs.
|
||||
@@ -0,0 +1,39 @@
|
||||
# Authentication and Signature V4
|
||||
|
||||
I.R.I.S. accepts AWS Signature Version 4 in either form:
|
||||
|
||||
- an `Authorization: AWS4-HMAC-SHA256 …` header with `x-amz-date`; or
|
||||
- a presigned request with `X-Amz-Algorithm`, `X-Amz-Credential`, `X-Amz-Date`, `X-Amz-SignedHeaders`, and `X-Amz-Signature` query parameters.
|
||||
|
||||
The `Credential` access-key ID must equal the Worker's `ACCESS_KEY`; the signing key is derived from `SECRET_KEY`, `REGION`, service `s3`, and the `YYYYMMDD` request date.
|
||||
|
||||
## Presigned URL expiry
|
||||
|
||||
Query-authenticated requests require `X-Amz-Expires` from 1 through 604800 seconds (seven days). The Worker returns `403 AccessDenied` with `Request has expired` when it is missing, invalid, out of range, or its signed timestamp plus expiry is in the past.
|
||||
|
||||
BFFs should normally use a much shorter period such as five minutes.
|
||||
|
||||
## Header-authenticated requests
|
||||
|
||||
The Worker requires a valid `x-amz-date` and rejects a request when the server time differs from it by more than 15 minutes. This returns `403 RequestTimeTooSkewed`.
|
||||
|
||||
## Canonical request behavior
|
||||
|
||||
The canonical request is built from:
|
||||
|
||||
1. method;
|
||||
2. URL path;
|
||||
3. sorted query string (excluding `X-Amz-Signature` for presigned URLs);
|
||||
4. the signed-header list and normalized values;
|
||||
5. the signed-header list; and
|
||||
6. `x-amz-content-sha256`, defaulting to `UNSIGNED-PAYLOAD`.
|
||||
|
||||
I.R.I.S. deliberately canonicalizes a signed `accept-encoding` header to `identity`. Cloudflare can rewrite the received value at the edge; S3 SDKs that sign this header use `identity` for this reason. Browser code must not sign `accept-encoding` because browser networking controls it.
|
||||
|
||||
Payload hashes are **not** verified. Browser and BFF clients should use `x-amz-content-sha256: UNSIGNED-PAYLOAD`; this is a deliberate streaming limitation, not an integrity guarantee.
|
||||
|
||||
## Public-read buckets
|
||||
|
||||
Buckets listed in `PUBLIC_READ_BUCKETS` permit unsigned `GET` and `HEAD` requests. All write operations still require valid Signature V4 authentication. `PUBLIC_READ_BUCKETS` must be a subset of `ALLOWED_BUCKETS`.
|
||||
|
||||
For a tested signing reference, see the `signed()` helper in [`test/s3.test.ts`](../test/s3.test.ts).
|
||||
@@ -0,0 +1,95 @@
|
||||
import { AbortMultipartUploadCommand, CompleteMultipartUploadCommand, CreateMultipartUploadCommand, PutObjectCommand, S3Client, UploadPartCommand } from "@aws-sdk/client-s3";
|
||||
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
|
||||
|
||||
interface Env {
|
||||
IRIS_ENDPOINT: string;
|
||||
IRIS_ACCESS_KEY: string;
|
||||
IRIS_SECRET_KEY: string;
|
||||
IRIS_REGION: string;
|
||||
IRIS_BUCKET: string;
|
||||
}
|
||||
|
||||
interface PresignRequest {
|
||||
key: string;
|
||||
contentType?: string;
|
||||
operation?: "put" | "createMultipart" | "uploadPart" | "completeMultipart" | "abortMultipart";
|
||||
uploadId?: string;
|
||||
partNumber?: number;
|
||||
parts?: Array<{ ETag: string; PartNumber: number }>;
|
||||
}
|
||||
|
||||
function escapeXml(value: string): string {
|
||||
return value.replace(/&/g, "&").replace(/</g, "<").replace(/>/g, ">").replace(/"/g, """).replace(/'/g, "'");
|
||||
}
|
||||
|
||||
function completionXml(parts: Array<{ ETag: string; PartNumber: number }> | undefined): string | null {
|
||||
if (!parts?.length || parts.some((part, index) => !Number.isInteger(part.PartNumber) || part.PartNumber !== index + 1 || typeof part.ETag !== "string" || part.ETag.length === 0)) return null;
|
||||
return `<CompleteMultipartUpload>${parts.map((part) => `<Part><PartNumber>${part.PartNumber}</PartNumber><ETag>${escapeXml(part.ETag)}</ETag></Part>`).join("")}</CompleteMultipartUpload>`;
|
||||
}
|
||||
|
||||
function json(data: unknown, status = 200): Response {
|
||||
return Response.json(data, { status, headers: { "Cache-Control": "no-store" } });
|
||||
}
|
||||
|
||||
function s3(env: Env): S3Client {
|
||||
return new S3Client({
|
||||
endpoint: env.IRIS_ENDPOINT,
|
||||
region: env.IRIS_REGION,
|
||||
forcePathStyle: true,
|
||||
credentials: { accessKeyId: env.IRIS_ACCESS_KEY, secretAccessKey: env.IRIS_SECRET_KEY },
|
||||
});
|
||||
}
|
||||
|
||||
function validKey(key: unknown): key is string {
|
||||
return typeof key === "string" && key.length > 0 && key.length <= 1024 && !key.startsWith("/") && !key.split("/").includes("..");
|
||||
}
|
||||
|
||||
export default {
|
||||
async fetch(request: Request, env: Env): Promise<Response> {
|
||||
if (request.method !== "POST" || new URL(request.url).pathname !== "/presign") return new Response("Not found", { status: 404 });
|
||||
|
||||
let input: PresignRequest;
|
||||
try {
|
||||
input = await request.json<PresignRequest>();
|
||||
} catch {
|
||||
return json({ error: "Expected JSON body" }, 400);
|
||||
}
|
||||
if (!validKey(input.key)) return json({ error: "Invalid object key" }, 400);
|
||||
|
||||
const client = s3(env);
|
||||
const operation = input.operation ?? "put";
|
||||
const expiresIn = 5 * 60;
|
||||
|
||||
if (operation === "put") {
|
||||
const contentType = input.contentType || "application/octet-stream";
|
||||
const url = await getSignedUrl(client, new PutObjectCommand({ Bucket: env.IRIS_BUCKET, Key: input.key, ContentType: contentType }), { expiresIn });
|
||||
return json({ url, method: "PUT", headers: { "Content-Type": contentType }, expiresIn });
|
||||
}
|
||||
|
||||
if (operation === "createMultipart") {
|
||||
const contentType = input.contentType || "application/octet-stream";
|
||||
const url = await getSignedUrl(client, new CreateMultipartUploadCommand({ Bucket: env.IRIS_BUCKET, Key: input.key, ContentType: contentType }), { expiresIn });
|
||||
return json({ url, method: "POST", headers: { "Content-Type": contentType }, expiresIn });
|
||||
}
|
||||
|
||||
if (!input.uploadId) return json({ error: "uploadId is required" }, 400);
|
||||
if (operation === "uploadPart") {
|
||||
const partNumber = input.partNumber;
|
||||
if (!Number.isInteger(partNumber) || partNumber === undefined || partNumber < 1 || partNumber > 10_000) return json({ error: "partNumber must be 1 through 10000" }, 400);
|
||||
const url = await getSignedUrl(client, new UploadPartCommand({ Bucket: env.IRIS_BUCKET, Key: input.key, UploadId: input.uploadId, PartNumber: partNumber }), { expiresIn });
|
||||
return json({ url, method: "PUT", expiresIn });
|
||||
}
|
||||
if (operation === "completeMultipart") {
|
||||
const body = completionXml(input.parts);
|
||||
if (!body) return json({ error: "parts must have sequential PartNumber values beginning at 1 and non-empty ETag values" }, 400);
|
||||
const url = await getSignedUrl(client, new CompleteMultipartUploadCommand({ Bucket: env.IRIS_BUCKET, Key: input.key, UploadId: input.uploadId, MultipartUpload: { Parts: input.parts } }), { expiresIn });
|
||||
return json({ url, method: "POST", headers: { "Content-Type": "application/xml" }, body, expiresIn });
|
||||
}
|
||||
if (operation === "abortMultipart") {
|
||||
const url = await getSignedUrl(client, new AbortMultipartUploadCommand({ Bucket: env.IRIS_BUCKET, Key: input.key, UploadId: input.uploadId }), { expiresIn });
|
||||
return json({ url, method: "DELETE", expiresIn });
|
||||
}
|
||||
|
||||
return json({ error: "Unsupported operation" }, 400);
|
||||
},
|
||||
} satisfies ExportedHandler<Env>;
|
||||
@@ -0,0 +1,48 @@
|
||||
interface PresignResponse {
|
||||
url: string;
|
||||
method: string;
|
||||
headers?: Record<string, string>;
|
||||
body?: string;
|
||||
}
|
||||
|
||||
async function presign(body: Record<string, unknown>): Promise<PresignResponse> {
|
||||
const response = await fetch("/presign", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
if (!response.ok) throw new Error(await response.text());
|
||||
return response.json<PresignResponse>();
|
||||
}
|
||||
|
||||
function uploadIdFromXml(xml: string): string {
|
||||
const value = /<UploadId>([^<]+)<\/UploadId>/.exec(xml)?.[1];
|
||||
if (!value) throw new Error("CreateMultipartUpload did not return UploadId");
|
||||
return value;
|
||||
}
|
||||
|
||||
/** Uploads parts in order. I.R.I.S. does not allow parallel or out-of-order part uploads. */
|
||||
export async function uploadLargeFile(file: File, key: string, partSize = 16 * 1024 * 1024): Promise<void> {
|
||||
const contentType = file.type || "application/octet-stream";
|
||||
const create = await presign({ key, contentType, operation: "createMultipart" });
|
||||
const createResponse = await fetch(create.url, { method: create.method, headers: create.headers });
|
||||
if (!createResponse.ok) throw new Error(await createResponse.text());
|
||||
const uploadId = uploadIdFromXml(await createResponse.text());
|
||||
|
||||
const parts: Array<{ ETag: string; PartNumber: number }> = [];
|
||||
for (let index = 0, offset = 0; offset < file.size; index++, offset += partSize) {
|
||||
const partNumber = index + 1;
|
||||
const body = file.slice(offset, Math.min(offset + partSize, file.size));
|
||||
const signed = await presign({ key, uploadId, partNumber, operation: "uploadPart" });
|
||||
const response = await fetch(signed.url, { method: signed.method, body });
|
||||
if (!response.ok) throw new Error(`Part ${partNumber} failed: ${await response.text()}`);
|
||||
const etag = response.headers.get("ETag");
|
||||
if (!etag) throw new Error(`Part ${partNumber} did not return ETag; check CORS_ALLOWED_ORIGINS`);
|
||||
parts.push({ ETag: etag, PartNumber: partNumber });
|
||||
}
|
||||
|
||||
const complete = await presign({ key, uploadId, parts, operation: "completeMultipart" });
|
||||
if (!complete.body) throw new Error("BFF did not return the completion XML body");
|
||||
const completeResponse = await fetch(complete.url, { method: complete.method, headers: complete.headers, body: complete.body });
|
||||
if (!completeResponse.ok) throw new Error(await completeResponse.text());
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
interface PresignedPut {
|
||||
url: string;
|
||||
method: "PUT";
|
||||
headers: { "Content-Type": string };
|
||||
expiresIn: number;
|
||||
}
|
||||
|
||||
async function getPresignedPut(key: string, contentType: string): Promise<PresignedPut> {
|
||||
const response = await fetch("/presign", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ key, contentType, operation: "put" }),
|
||||
});
|
||||
if (!response.ok) throw new Error(await response.text());
|
||||
return response.json<PresignedPut>();
|
||||
}
|
||||
|
||||
export async function uploadFile(file: File, key: string): Promise<string | null> {
|
||||
const signed = await getPresignedPut(key, file.type || "application/octet-stream");
|
||||
const response = await fetch(signed.url, {
|
||||
method: signed.method,
|
||||
headers: signed.headers,
|
||||
body: file,
|
||||
});
|
||||
if (!response.ok) throw new Error(`Upload failed: ${await response.text()}`);
|
||||
|
||||
// Requires CORS_ALLOWED_ORIGINS to include this page's exact origin.
|
||||
return response.headers.get("ETag");
|
||||
}
|
||||
@@ -0,0 +1,98 @@
|
||||
# Frontend integration guide
|
||||
|
||||
## Architecture
|
||||
|
||||
Use a backend-for-frontend (BFF) as the trust boundary:
|
||||
|
||||
```text
|
||||
Browser → BFF (holds ACCESS_KEY + SECRET_KEY, issues presigned URLs) → I.R.I.S. Worker → Google Drive
|
||||
```
|
||||
|
||||
Never put `SECRET_KEY` in browser code, a static site, or a public environment variable. The browser receives only a short-lived presigned URL for one method and object key.
|
||||
|
||||
## Enable browser access
|
||||
|
||||
Configure the Worker with an explicit allow-list of web origins:
|
||||
|
||||
```ini
|
||||
CORS_ALLOWED_ORIGINS=https://app.example.com,http://localhost:5173
|
||||
```
|
||||
|
||||
Set it to `*` only for a genuinely public, credential-free integration. Leaving it unset emits no CORS headers. I.R.I.S. exposes `ETag`, `Content-Range`, `Content-Length`, `Last-Modified`, `Accept-Ranges`, and `x-amz-request-id`, so browser code can inspect the headers that form the effective S3 response payload.
|
||||
|
||||
### Server-to-server compatibility
|
||||
|
||||
CORS does not restrict non-browser clients. `Access-Control-*` headers are emitted only when a request has an allowed `Origin`; a CLI, Dokploy, rclone, aws-cli, or other service request with no `Origin` keeps its normal S3 status and body. With an exact origin allow-list, responses also include `Vary: Origin`, including requests without an `Origin`, so shared caches cannot replay a no-origin object response to a browser. S3 clients ignore this cache-control metadata.
|
||||
|
||||
An `OPTIONS` request with no `Origin`, or a preflight from a disallowed origin, receives the same bare `204` as a CORS-disabled Worker. A disallowed `Origin` never changes the underlying S3 result—it simply receives no `Access-Control-*` headers.
|
||||
|
||||
## Presign in the BFF
|
||||
|
||||
Use `@aws-sdk/s3-request-presigner` and send a limited expiry. I.R.I.S. validates `X-Amz-Expires` and rejects URLs after their signing time plus that value.
|
||||
|
||||
```ts
|
||||
import { PutObjectCommand, S3Client } from "@aws-sdk/client-s3";
|
||||
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
|
||||
|
||||
const client = new S3Client({
|
||||
endpoint: env.IRIS_ENDPOINT,
|
||||
region: env.IRIS_REGION,
|
||||
credentials: {
|
||||
accessKeyId: env.IRIS_ACCESS_KEY,
|
||||
secretAccessKey: env.IRIS_SECRET_KEY,
|
||||
},
|
||||
forcePathStyle: true,
|
||||
});
|
||||
|
||||
const url = await getSignedUrl(
|
||||
client,
|
||||
new PutObjectCommand({
|
||||
Bucket: "assets",
|
||||
Key: "uploads/avatar.png",
|
||||
ContentType: "image/png",
|
||||
}),
|
||||
{ expiresIn: 300 },
|
||||
);
|
||||
```
|
||||
|
||||
Keep the `ContentType` in the command and send precisely that same `Content-Type` from the browser. It becomes the Drive `mimeType`. A non-simple content type causes a preflight request, which is why the Worker CORS configuration is necessary.
|
||||
|
||||
## Upload one object
|
||||
|
||||
Use `Blob`, `File`, `ArrayBuffer`, or a typed array as the body. This lets the browser provide a known body length. Do **not** use a `ReadableStream` unless the BFF signs `x-amz-decoded-content-length` and the browser sends it: multipart part uploads require `Content-Length` or `x-amz-decoded-content-length`.
|
||||
|
||||
```ts
|
||||
const response = await fetch(presignedUrl, {
|
||||
method: "PUT",
|
||||
headers: { "Content-Type": file.type || "application/octet-stream" },
|
||||
body: file,
|
||||
});
|
||||
|
||||
if (!response.ok) throw new Error(await response.text());
|
||||
const etag = response.headers.get("ETag");
|
||||
```
|
||||
|
||||
`ETag` is readable only when the request `Origin` is configured in `CORS_ALLOWED_ORIGINS`.
|
||||
|
||||
## Multipart upload
|
||||
|
||||
Cloudflare request limits mean a single PUT is best kept below roughly 100 MB. For larger files, have the BFF initiate the upload, issue a presigned URL for each part, and complete the upload after the browser returns all ETags.
|
||||
|
||||
Use 16 MB parts as a practical baseline. Upload parts **strictly sequentially starting at 1**. I.R.I.S. waits up to 20 seconds for an earlier part; out-of-order work then receives `503 SlowDown`. Requests more than 64 part numbers ahead are rejected immediately.
|
||||
|
||||
1. BFF signs `POST /bucket/key?uploads` and starts the multipart upload.
|
||||
2. BFF issues a signed `PUT` URL for part 1; browser uploads it and records the `ETag`.
|
||||
3. Repeat one part at a time for parts 2 through $n$.
|
||||
4. BFF signs `POST /bucket/key?uploadId=…`; browser sends the `CompleteMultipartUpload` XML body containing the sequential part numbers and ETags.
|
||||
|
||||
See [`examples/browser-multipart.ts`](./examples/browser-multipart.ts) for a complete browser-side loop.
|
||||
|
||||
## Build a file browser
|
||||
|
||||
List with `prefix` and `delimiter=/`:
|
||||
|
||||
```text
|
||||
GET /assets?prefix=photos/&delimiter=/
|
||||
```
|
||||
|
||||
Files appear in `Contents`; immediate child folders appear as `CommonPrefixes`. Google Drive directories are physical folders, not zero-byte marker objects. Read [limitations](./limitations.md) before designing pagination or rename/move features.
|
||||
@@ -0,0 +1,35 @@
|
||||
# Limitations
|
||||
|
||||
I.R.I.S. is a narrow S3 compatibility layer. Design frontend features around the behavior below.
|
||||
|
||||
## Listing is not pageable
|
||||
|
||||
`max-keys`, `continuation-token`, and `marker` are ignored. I.R.I.S. scans at most 5000 Google Drive nodes and then sets `IsTruncated=true`; it does not provide a continuation token, so additional results cannot be requested. The XML value `MaxKeys=1000` is a fixed compatibility value, not the real scan limit.
|
||||
|
||||
Use narrow prefixes and `delimiter=/` for a file browser. Do not build an infinite-scrolling full-drive browser.
|
||||
|
||||
## No server-side rename or move
|
||||
|
||||
`CopyObject` and `UploadPartCopy` return `501 NotImplemented`. There is no server-side rename or move because both require copying. Implement rename/move as download, re-upload under the target key, then delete the original—subject to bandwidth, time, and error handling.
|
||||
|
||||
## Unsupported S3 features
|
||||
|
||||
- `ListBuckets`, `CreateBucket`, and `DeleteBucket`
|
||||
- batch `DeleteObjects`
|
||||
- object versioning, ACLs, object tags, lifecycle rules, and `x-amz-meta-*` user metadata
|
||||
- CopyObject and UploadPartCopy
|
||||
- discovering active multipart uploads: `ListMultipartUploads` always reports an empty list
|
||||
|
||||
Bucket `?acl`, `?versioning`, and `?location` requests are currently treated as regular bucket listing requests rather than errors. Do not rely on this accidental compatibility.
|
||||
|
||||
## Directories and objects
|
||||
|
||||
Directories are physical Google Drive folders created from object-key path segments. I.R.I.S. does not create zero-byte directory marker objects. An empty prefix can therefore exist as a Drive folder without an S3 marker object.
|
||||
|
||||
## Multipart behavior
|
||||
|
||||
Parts must be uploaded strictly in consecutive order beginning at part 1. Parts are immutable after they have committed. An incomplete upload expires after 24 hours. `ETAG_STYLE=md5` returns Google Drive's file MD5 for completed multipart objects; `ETAG_STYLE=multipart` is available only for clients that require an S3-style composite ETag.
|
||||
|
||||
## Quotas and plan limits
|
||||
|
||||
Google Drive's free tier has 15 GB total storage and a 750 GB/day upload limit. Cloudflare applies request-size limits before a request reaches the Worker; use multipart uploads for large browser uploads.
|
||||
@@ -0,0 +1,391 @@
|
||||
openapi: 3.1.0
|
||||
info:
|
||||
title: I.R.I.S. Google Drive S3 API
|
||||
version: 1.0.0
|
||||
description: |
|
||||
An intentionally small S3-compatible object-storage API backed by Google Drive.
|
||||
Authenticate every non-public request with AWS Signature Version 4. Browser clients
|
||||
should receive short-lived presigned URLs from a backend-for-frontend, never credentials.
|
||||
servers:
|
||||
- url: https://{worker-host}
|
||||
variables:
|
||||
worker-host:
|
||||
default: your-worker.example.workers.dev
|
||||
security:
|
||||
- sigv4: []
|
||||
tags:
|
||||
- name: Objects
|
||||
- name: Buckets
|
||||
- name: Multipart uploads
|
||||
paths:
|
||||
/{bucket}:
|
||||
parameters:
|
||||
- $ref: '#/components/parameters/Bucket'
|
||||
get:
|
||||
tags: [Buckets]
|
||||
operationId: listObjects
|
||||
summary: List objects in a bucket
|
||||
description: List objects recursively, or use `delimiter=/` to return immediate folders as `CommonPrefixes`.
|
||||
parameters:
|
||||
- $ref: '#/components/parameters/Prefix'
|
||||
- $ref: '#/components/parameters/Delimiter'
|
||||
responses:
|
||||
'200':
|
||||
description: Object listing.
|
||||
content:
|
||||
application/xml:
|
||||
schema: { $ref: '#/components/schemas/ListBucketResult' }
|
||||
'403': { $ref: '#/components/responses/AccessDenied' }
|
||||
'405': { $ref: '#/components/responses/MethodNotAllowed' }
|
||||
head:
|
||||
tags: [Buckets]
|
||||
operationId: headBucket
|
||||
summary: Check bucket access
|
||||
responses:
|
||||
'200': { description: The bucket is configured and accessible. }
|
||||
'403': { $ref: '#/components/responses/AccessDenied' }
|
||||
'405': { $ref: '#/components/responses/MethodNotAllowed' }
|
||||
/{bucket}?uploads:
|
||||
parameters:
|
||||
- $ref: '#/components/parameters/Bucket'
|
||||
get:
|
||||
tags: [Multipart uploads]
|
||||
operationId: listMultipartUploads
|
||||
summary: List multipart uploads
|
||||
description: Always returns an empty list; active multipart uploads cannot be enumerated.
|
||||
responses:
|
||||
'200':
|
||||
description: Empty multipart upload listing.
|
||||
content:
|
||||
application/xml:
|
||||
schema: { $ref: '#/components/schemas/ListMultipartUploadsResult' }
|
||||
'403': { $ref: '#/components/responses/AccessDenied' }
|
||||
/{bucket}/{key}:
|
||||
parameters:
|
||||
- $ref: '#/components/parameters/Bucket'
|
||||
- $ref: '#/components/parameters/Key'
|
||||
put:
|
||||
tags: [Objects]
|
||||
operationId: putObject
|
||||
summary: Create or replace an object
|
||||
parameters:
|
||||
- $ref: '#/components/parameters/ContentType'
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/octet-stream:
|
||||
schema: { type: string, format: binary }
|
||||
'*/*':
|
||||
schema: { type: string, format: binary }
|
||||
responses:
|
||||
'200':
|
||||
description: Object uploaded.
|
||||
headers:
|
||||
ETag: { $ref: '#/components/headers/ETag' }
|
||||
'403': { $ref: '#/components/responses/AccessDenied' }
|
||||
'501': { $ref: '#/components/responses/NotImplemented' }
|
||||
get:
|
||||
tags: [Objects]
|
||||
operationId: getObject
|
||||
summary: Download an object
|
||||
parameters:
|
||||
- name: Range
|
||||
in: header
|
||||
schema: { type: string, example: bytes=0-1023 }
|
||||
responses:
|
||||
'200':
|
||||
description: Full object data.
|
||||
headers:
|
||||
ETag: { $ref: '#/components/headers/ETag' }
|
||||
Content-Length: { $ref: '#/components/headers/ContentLength' }
|
||||
Last-Modified: { $ref: '#/components/headers/LastModified' }
|
||||
Accept-Ranges: { schema: { type: string, example: bytes } }
|
||||
content:
|
||||
application/octet-stream:
|
||||
schema: { type: string, format: binary }
|
||||
'206':
|
||||
description: Requested byte range.
|
||||
headers:
|
||||
Content-Range: { schema: { type: string, example: bytes 0-1023/4096 } }
|
||||
Content-Length: { $ref: '#/components/headers/ContentLength' }
|
||||
content:
|
||||
application/octet-stream:
|
||||
schema: { type: string, format: binary }
|
||||
'403': { $ref: '#/components/responses/AccessDenied' }
|
||||
'404': { $ref: '#/components/responses/NoSuchKey' }
|
||||
head:
|
||||
tags: [Objects]
|
||||
operationId: headObject
|
||||
summary: Get object metadata
|
||||
responses:
|
||||
'200':
|
||||
description: Object metadata in headers.
|
||||
headers:
|
||||
ETag: { $ref: '#/components/headers/ETag' }
|
||||
Content-Length: { $ref: '#/components/headers/ContentLength' }
|
||||
Last-Modified: { $ref: '#/components/headers/LastModified' }
|
||||
Accept-Ranges: { schema: { type: string, example: bytes } }
|
||||
'403': { $ref: '#/components/responses/AccessDenied' }
|
||||
'404': { $ref: '#/components/responses/NoSuchKey' }
|
||||
delete:
|
||||
tags: [Objects]
|
||||
operationId: deleteObject
|
||||
summary: Delete an object
|
||||
responses:
|
||||
'204': { description: Object deleted. }
|
||||
'403': { $ref: '#/components/responses/AccessDenied' }
|
||||
'404': { $ref: '#/components/responses/NoSuchKey' }
|
||||
/{bucket}/{key}?uploads:
|
||||
parameters:
|
||||
- $ref: '#/components/parameters/Bucket'
|
||||
- $ref: '#/components/parameters/Key'
|
||||
post:
|
||||
tags: [Multipart uploads]
|
||||
operationId: createMultipartUpload
|
||||
summary: Initiate a multipart upload
|
||||
parameters:
|
||||
- $ref: '#/components/parameters/ContentType'
|
||||
responses:
|
||||
'200':
|
||||
description: Upload initiated.
|
||||
content:
|
||||
application/xml:
|
||||
schema: { $ref: '#/components/schemas/InitiateMultipartUploadResult' }
|
||||
'403': { $ref: '#/components/responses/AccessDenied' }
|
||||
'501': { $ref: '#/components/responses/NotImplemented' }
|
||||
/{bucket}/{key}?uploadId={uploadId}&partNumber={partNumber}:
|
||||
parameters:
|
||||
- $ref: '#/components/parameters/Bucket'
|
||||
- $ref: '#/components/parameters/Key'
|
||||
- $ref: '#/components/parameters/UploadId'
|
||||
- $ref: '#/components/parameters/PartNumber'
|
||||
put:
|
||||
tags: [Multipart uploads]
|
||||
operationId: uploadPart
|
||||
summary: Upload one multipart part
|
||||
description: Parts must arrive strictly sequentially beginning with part number 1.
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/octet-stream:
|
||||
schema: { type: string, format: binary }
|
||||
responses:
|
||||
'200':
|
||||
description: Part accepted.
|
||||
headers:
|
||||
ETag: { $ref: '#/components/headers/ETag' }
|
||||
'400': { $ref: '#/components/responses/InvalidArgument' }
|
||||
'403': { $ref: '#/components/responses/AccessDenied' }
|
||||
'404': { $ref: '#/components/responses/NoSuchUpload' }
|
||||
'503': { $ref: '#/components/responses/SlowDown' }
|
||||
/{bucket}/{key}?uploadId={uploadId}:
|
||||
parameters:
|
||||
- $ref: '#/components/parameters/Bucket'
|
||||
- $ref: '#/components/parameters/Key'
|
||||
- $ref: '#/components/parameters/UploadId'
|
||||
post:
|
||||
tags: [Multipart uploads]
|
||||
operationId: completeMultipartUpload
|
||||
summary: Complete a multipart upload
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/xml:
|
||||
schema: { $ref: '#/components/schemas/CompleteMultipartUpload' }
|
||||
responses:
|
||||
'200':
|
||||
description: Upload completed.
|
||||
content:
|
||||
application/xml:
|
||||
schema: { $ref: '#/components/schemas/CompleteMultipartUploadResult' }
|
||||
'400': { $ref: '#/components/responses/InvalidArgument' }
|
||||
'403': { $ref: '#/components/responses/AccessDenied' }
|
||||
'404': { $ref: '#/components/responses/NoSuchUpload' }
|
||||
get:
|
||||
tags: [Multipart uploads]
|
||||
operationId: listParts
|
||||
summary: List uploaded parts
|
||||
parameters:
|
||||
- name: part-number-marker
|
||||
in: query
|
||||
schema: { type: integer, minimum: 0, default: 0 }
|
||||
- name: max-parts
|
||||
in: query
|
||||
schema: { type: integer, minimum: 1, maximum: 1000, default: 1000 }
|
||||
responses:
|
||||
'200':
|
||||
description: Current part list.
|
||||
content:
|
||||
application/xml:
|
||||
schema: { $ref: '#/components/schemas/ListPartsResult' }
|
||||
'400': { $ref: '#/components/responses/InvalidArgument' }
|
||||
'403': { $ref: '#/components/responses/AccessDenied' }
|
||||
'404': { $ref: '#/components/responses/NoSuchUpload' }
|
||||
delete:
|
||||
tags: [Multipart uploads]
|
||||
operationId: abortMultipartUpload
|
||||
summary: Abort a multipart upload
|
||||
responses:
|
||||
'204': { description: Upload aborted. }
|
||||
'403': { $ref: '#/components/responses/AccessDenied' }
|
||||
'404': { $ref: '#/components/responses/NoSuchUpload' }
|
||||
components:
|
||||
securitySchemes:
|
||||
sigv4:
|
||||
type: apiKey
|
||||
in: header
|
||||
name: Authorization
|
||||
description: AWS Signature Version 4 header authentication or equivalent `X-Amz-*` presigned query parameters.
|
||||
parameters:
|
||||
Bucket:
|
||||
name: bucket
|
||||
in: path
|
||||
required: true
|
||||
schema: { type: string }
|
||||
Key:
|
||||
name: key
|
||||
in: path
|
||||
required: true
|
||||
schema: { type: string }
|
||||
Prefix:
|
||||
name: prefix
|
||||
in: query
|
||||
schema: { type: string, default: '' }
|
||||
Delimiter:
|
||||
name: delimiter
|
||||
in: query
|
||||
schema: { type: string, example: / }
|
||||
UploadId:
|
||||
name: uploadId
|
||||
in: query
|
||||
required: true
|
||||
schema: { type: string }
|
||||
PartNumber:
|
||||
name: partNumber
|
||||
in: query
|
||||
required: true
|
||||
schema: { type: integer, minimum: 1, maximum: 10000 }
|
||||
ContentType:
|
||||
name: Content-Type
|
||||
in: header
|
||||
schema: { type: string, example: application/octet-stream }
|
||||
headers:
|
||||
ETag:
|
||||
schema: { type: string, example: '"d41d8cd98f00b204e9800998ecf8427e"' }
|
||||
ContentLength:
|
||||
schema: { type: integer, minimum: 0 }
|
||||
LastModified:
|
||||
schema: { type: string, format: date-time }
|
||||
responses:
|
||||
AccessDenied:
|
||||
description: Authentication failed, the access key is not recognized, the presigned URL expired, or bucket access is denied.
|
||||
content: { application/xml: { schema: { $ref: '#/components/schemas/S3Error' } } }
|
||||
NoSuchKey:
|
||||
description: The object does not exist.
|
||||
content: { application/xml: { schema: { $ref: '#/components/schemas/S3Error' } } }
|
||||
NoSuchUpload:
|
||||
description: The multipart upload does not exist.
|
||||
content: { application/xml: { schema: { $ref: '#/components/schemas/S3Error' } } }
|
||||
InvalidArgument:
|
||||
description: A required parameter or body is invalid.
|
||||
content: { application/xml: { schema: { $ref: '#/components/schemas/S3Error' } } }
|
||||
NotImplemented:
|
||||
description: This S3 operation is not supported.
|
||||
content: { application/xml: { schema: { $ref: '#/components/schemas/S3Error' } } }
|
||||
SlowDown:
|
||||
description: Parts are out of sequence or the upload is busy; retry later.
|
||||
content: { application/xml: { schema: { $ref: '#/components/schemas/S3Error' } } }
|
||||
MethodNotAllowed:
|
||||
description: The method/path combination is unsupported.
|
||||
content: { application/xml: { schema: { $ref: '#/components/schemas/S3Error' } } }
|
||||
schemas:
|
||||
S3Error:
|
||||
type: object
|
||||
xml: { name: Error }
|
||||
required: [Code, Message, RequestId]
|
||||
properties:
|
||||
Code: { type: string, xml: { name: Code } }
|
||||
Message: { type: string, xml: { name: Message } }
|
||||
Resource: { type: string, xml: { name: Resource } }
|
||||
RequestId: { type: string, xml: { name: RequestId } }
|
||||
Object:
|
||||
type: object
|
||||
xml: { name: Contents }
|
||||
properties:
|
||||
Key: { type: string }
|
||||
LastModified: { type: string, format: date-time }
|
||||
ETag: { type: string }
|
||||
Size: { type: integer }
|
||||
StorageClass: { type: string, example: STANDARD }
|
||||
ListBucketResult:
|
||||
type: object
|
||||
xml: { name: ListBucketResult, namespace: 'http://s3.amazonaws.com/doc/2006-03-01/' }
|
||||
properties:
|
||||
Name: { type: string }
|
||||
Prefix: { type: string }
|
||||
Delimiter: { type: string }
|
||||
MaxKeys: { type: integer, example: 1000 }
|
||||
IsTruncated: { type: boolean }
|
||||
Contents: { type: array, items: { $ref: '#/components/schemas/Object' }, xml: { wrapped: false } }
|
||||
InitiateMultipartUploadResult:
|
||||
type: object
|
||||
xml: { name: InitiateMultipartUploadResult, namespace: 'http://s3.amazonaws.com/doc/2006-03-01/' }
|
||||
required: [Bucket, Key, UploadId]
|
||||
properties:
|
||||
Bucket: { type: string }
|
||||
Key: { type: string }
|
||||
UploadId: { type: string }
|
||||
CompleteMultipartUpload:
|
||||
type: object
|
||||
xml: { name: CompleteMultipartUpload }
|
||||
required: [Part]
|
||||
properties:
|
||||
Part:
|
||||
type: array
|
||||
minItems: 1
|
||||
xml: { wrapped: false }
|
||||
items:
|
||||
type: object
|
||||
xml: { name: Part }
|
||||
required: [PartNumber, ETag]
|
||||
properties:
|
||||
PartNumber: { type: integer }
|
||||
ETag: { type: string }
|
||||
CompleteMultipartUploadResult:
|
||||
type: object
|
||||
xml: { name: CompleteMultipartUploadResult, namespace: 'http://s3.amazonaws.com/doc/2006-03-01/' }
|
||||
properties:
|
||||
Location: { type: string }
|
||||
Bucket: { type: string }
|
||||
Key: { type: string }
|
||||
ETag: { type: string }
|
||||
ListPartsResult:
|
||||
type: object
|
||||
xml: { name: ListPartsResult, namespace: 'http://s3.amazonaws.com/doc/2006-03-01/' }
|
||||
properties:
|
||||
Bucket: { type: string }
|
||||
Key: { type: string }
|
||||
UploadId: { type: string }
|
||||
NextPartNumberMarker: { type: integer }
|
||||
IsTruncated: { type: boolean }
|
||||
Part:
|
||||
type: array
|
||||
xml: { wrapped: false }
|
||||
items:
|
||||
type: object
|
||||
properties:
|
||||
PartNumber: { type: integer }
|
||||
LastModified: { type: string, format: date-time }
|
||||
ETag: { type: string }
|
||||
Size: { type: integer }
|
||||
ListMultipartUploadsResult:
|
||||
type: object
|
||||
xml: { name: ListMultipartUploadsResult, namespace: 'http://s3.amazonaws.com/doc/2006-03-01/' }
|
||||
properties:
|
||||
Bucket: { type: string }
|
||||
KeyMarker: { type: string }
|
||||
UploadIdMarker: { type: string }
|
||||
NextKeyMarker: { type: string }
|
||||
NextUploadIdMarker: { type: string }
|
||||
MaxUploads: { type: integer, example: 1000 }
|
||||
IsTruncated: { type: boolean, example: false }
|
||||
@@ -20,9 +20,9 @@
|
||||
// Optional: --port 8976 (must not be in use; Desktop-app clients accept any
|
||||
// loopback port automatically, no need to register it in Google Cloud Console).
|
||||
|
||||
import { exec } from "node:child_process";
|
||||
import { randomBytes } from "node:crypto";
|
||||
import http from "node:http";
|
||||
import { exec } from "node:child_process";
|
||||
import { URL } from "node:url";
|
||||
|
||||
const SCOPE = "https://www.googleapis.com/auth/drive";
|
||||
@@ -137,10 +137,7 @@ async function main() {
|
||||
process.exit(1);
|
||||
}
|
||||
if (!tokenData.refresh_token) {
|
||||
console.error(
|
||||
"No refresh_token was returned. This usually means the account already has an active grant.\n" +
|
||||
"Revoke it at https://myaccount.google.com/permissions and re-run this script.",
|
||||
);
|
||||
console.error("No refresh_token was returned. This usually means the account already has an active grant.\n" + "Revoke it at https://myaccount.google.com/permissions and re-run this script.");
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
|
||||
+56
-6
@@ -1,5 +1,11 @@
|
||||
import type { S3ErrorCode } from "./s3-errors";
|
||||
import type { Env } from "./types";
|
||||
|
||||
const MAX_PRESIGN_EXPIRY_SECONDS = 7 * 24 * 60 * 60;
|
||||
const MAX_CLOCK_SKEW_MS = 15 * 60 * 1000;
|
||||
|
||||
export type VerifyResult = { ok: true } | { ok: false; code: S3ErrorCode; message?: string };
|
||||
|
||||
function encodeRFC3986(str: string): string {
|
||||
return encodeURIComponent(str).replace(/[!'()*]/g, (c) => `%${c.charCodeAt(0).toString(16).toUpperCase()}`);
|
||||
}
|
||||
@@ -84,8 +90,35 @@ async function createCanonicalRequest(request: Request, isQueryAuth: boolean): P
|
||||
return [method, canonicalUri, params, canonicalHeaders, signedHeaders, payloadHash].join("\n");
|
||||
}
|
||||
|
||||
function parseAmzDate(datetime: string): number | null {
|
||||
const match = /^(\d{4})(\d{2})(\d{2})T(\d{2})(\d{2})(\d{2})Z$/.exec(datetime);
|
||||
if (!match) return null;
|
||||
|
||||
const [, year, month, day, hour, minute, second] = match;
|
||||
const timestamp = Date.UTC(Number(year), Number(month) - 1, Number(day), Number(hour), Number(minute), Number(second));
|
||||
const parsed = new Date(timestamp);
|
||||
if (parsed.getUTCFullYear() !== Number(year) || parsed.getUTCMonth() !== Number(month) - 1 || parsed.getUTCDate() !== Number(day) || parsed.getUTCHours() !== Number(hour) || parsed.getUTCMinutes() !== Number(minute) || parsed.getUTCSeconds() !== Number(second)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return timestamp;
|
||||
}
|
||||
|
||||
function constantTimeEqual(left: string, right: string): boolean {
|
||||
if (left.length !== right.length) return false;
|
||||
let mismatch = 0;
|
||||
for (let index = 0; index < left.length; index++) {
|
||||
mismatch |= left.charCodeAt(index) ^ right.charCodeAt(index);
|
||||
}
|
||||
return mismatch === 0;
|
||||
}
|
||||
|
||||
function signatureMismatch(): VerifyResult {
|
||||
return { ok: false, code: "SignatureDoesNotMatch" };
|
||||
}
|
||||
|
||||
/** Verifies an AWS Signature V4 signature carried in either the Authorization header or presigned query params. */
|
||||
export async function verifySignature(request: Request, env: Env): Promise<boolean> {
|
||||
export async function verifySignature(request: Request, env: Env): Promise<VerifyResult> {
|
||||
const url = new URL(request.url);
|
||||
const headers = request.headers;
|
||||
|
||||
@@ -99,15 +132,32 @@ export async function verifySignature(request: Request, env: Env): Promise<boole
|
||||
algorithm = authHeader.split(" ")[0];
|
||||
}
|
||||
|
||||
if (!algorithm?.includes("AWS4-HMAC-SHA256")) {
|
||||
return false;
|
||||
if (algorithm !== "AWS4-HMAC-SHA256") {
|
||||
return signatureMismatch();
|
||||
}
|
||||
|
||||
const datetime = (isQueryAuth ? url.searchParams.get("X-Amz-Date") : headers.get("x-amz-date")) ?? "";
|
||||
if (!datetime) return isQueryAuth ? { ok: false, code: "AccessDenied", message: "Request has expired" } : signatureMismatch();
|
||||
const signedAt = parseAmzDate(datetime);
|
||||
if (signedAt === null) return isQueryAuth ? { ok: false, code: "AccessDenied", message: "Request has expired" } : signatureMismatch();
|
||||
|
||||
if (!datetime) return false;
|
||||
if (isQueryAuth) {
|
||||
const expires = url.searchParams.get("X-Amz-Expires");
|
||||
if (!expires || !/^\d+$/.test(expires)) return { ok: false, code: "AccessDenied", message: "Request has expired" };
|
||||
const expiresIn = Number(expires);
|
||||
if (!Number.isSafeInteger(expiresIn) || expiresIn < 1 || expiresIn > MAX_PRESIGN_EXPIRY_SECONDS || Date.now() > signedAt + expiresIn * 1000) {
|
||||
return { ok: false, code: "AccessDenied", message: "Request has expired" };
|
||||
}
|
||||
} else if (Math.abs(Date.now() - signedAt) > MAX_CLOCK_SKEW_MS) {
|
||||
return { ok: false, code: "RequestTimeTooSkewed" };
|
||||
}
|
||||
|
||||
const date = datetime.substring(0, 8);
|
||||
const credential = isQueryAuth ? (url.searchParams.get("X-Amz-Credential") ?? "") : (/Credential=([^,\s]+)/.exec(headers.get("Authorization") ?? "")?.[1] ?? "");
|
||||
const credentialParts = credential.split("/");
|
||||
if (credentialParts.length !== 5 || credentialParts[0] !== env.ACCESS_KEY || credentialParts.slice(1).join("/") !== `${date}/${env.REGION}/s3/aws4_request`) {
|
||||
return { ok: false, code: "AccessDenied" };
|
||||
}
|
||||
|
||||
const canonicalRequest = await createCanonicalRequest(request, isQueryAuth);
|
||||
const hashedCanonicalRequest = await sha256(canonicalRequest);
|
||||
@@ -124,9 +174,9 @@ export async function verifySignature(request: Request, env: Env): Promise<boole
|
||||
expectedSignature = url.searchParams.get("X-Amz-Signature") ?? "";
|
||||
} else {
|
||||
const authHeader = headers.get("Authorization") ?? "";
|
||||
const match = authHeader.match(/Signature=([a-f0-9]+)/);
|
||||
const match = authHeader.match(/Signature=([a-fA-F0-9]+)/);
|
||||
expectedSignature = match ? match[1] : "";
|
||||
}
|
||||
|
||||
return signatureHex === expectedSignature;
|
||||
return constantTimeEqual(signatureHex, expectedSignature) ? { ok: true } : signatureMismatch();
|
||||
}
|
||||
|
||||
+69
@@ -0,0 +1,69 @@
|
||||
import type { Env } from "./types";
|
||||
|
||||
const ALLOWED_METHODS = "GET, HEAD, PUT, POST, DELETE, OPTIONS";
|
||||
const DEFAULT_ALLOWED_HEADERS = "Authorization, Content-Type, Content-Length, Content-MD5, Content-Encoding, Range, x-amz-content-sha256, x-amz-copy-source, x-amz-date, x-amz-decoded-content-length, x-amz-mp-object-size, x-amz-security-token";
|
||||
const EXPOSED_HEADERS = "ETag, Content-Range, Content-Length, Last-Modified, Accept-Ranges, x-amz-request-id";
|
||||
|
||||
function configuredOrigins(env: Env): string[] {
|
||||
return (env.CORS_ALLOWED_ORIGINS ?? "")
|
||||
.split(",")
|
||||
.map((origin) => origin.trim())
|
||||
.filter(Boolean);
|
||||
}
|
||||
|
||||
function hasWildcardOrigin(env: Env): boolean {
|
||||
return env.CORS_ALLOWED_ORIGINS?.trim() === "*";
|
||||
}
|
||||
|
||||
function appendVary(headers: Headers, value: string): void {
|
||||
const existing = headers.get("Vary");
|
||||
if (!existing) {
|
||||
headers.set("Vary", value);
|
||||
return;
|
||||
}
|
||||
|
||||
if (!existing.split(",").some((part) => part.trim().toLowerCase() === value.toLowerCase())) {
|
||||
headers.set("Vary", `${existing}, ${value}`);
|
||||
}
|
||||
}
|
||||
|
||||
/** Returns the permitted value for Access-Control-Allow-Origin, or null when CORS is disabled or denied. */
|
||||
export function resolveOrigin(request: Request, env: Env): string | null {
|
||||
const origin = request.headers.get("Origin");
|
||||
if (!origin) return null;
|
||||
if (hasWildcardOrigin(env)) return "*";
|
||||
return configuredOrigins(env).includes(origin) ? origin : null;
|
||||
}
|
||||
|
||||
/** Builds an unauthenticated browser preflight response. */
|
||||
export function preflightResponse(request: Request, env: Env): Response {
|
||||
const headers = new Headers();
|
||||
const origin = resolveOrigin(request, env);
|
||||
|
||||
if (origin) {
|
||||
headers.set("Access-Control-Allow-Origin", origin);
|
||||
headers.set("Access-Control-Allow-Methods", ALLOWED_METHODS);
|
||||
headers.set("Access-Control-Allow-Headers", request.headers.get("Access-Control-Request-Headers") ?? DEFAULT_ALLOWED_HEADERS);
|
||||
headers.set("Access-Control-Max-Age", "86400");
|
||||
if (!hasWildcardOrigin(env)) appendVary(headers, "Origin");
|
||||
}
|
||||
|
||||
return new Response(null, { status: 204, headers });
|
||||
}
|
||||
|
||||
/** Adds CORS response headers without mutating the response supplied by the request handler. */
|
||||
export function withCors(response: Response, request: Request, env: Env): Response {
|
||||
const headers = new Headers(response.headers);
|
||||
|
||||
if (env.CORS_ALLOWED_ORIGINS?.trim() && !hasWildcardOrigin(env)) {
|
||||
appendVary(headers, "Origin");
|
||||
}
|
||||
|
||||
const origin = resolveOrigin(request, env);
|
||||
if (origin) {
|
||||
headers.set("Access-Control-Allow-Origin", origin);
|
||||
headers.set("Access-Control-Expose-Headers", EXPOSED_HEADERS);
|
||||
}
|
||||
|
||||
return new Response(response.body, { status: response.status, statusText: response.statusText, headers });
|
||||
}
|
||||
+36
@@ -0,0 +1,36 @@
|
||||
import openApiSpec from "../docs/openapi.yaml";
|
||||
|
||||
export const OPENAPI_PATH = "/openapi.yaml";
|
||||
export const DOCS_PATH = "/docs";
|
||||
|
||||
export function openApiResponse(): Response {
|
||||
return new Response(openApiSpec, {
|
||||
headers: {
|
||||
"Content-Type": "application/yaml; charset=utf-8",
|
||||
"Cache-Control": "public, max-age=300",
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
export function docsResponse(): Response {
|
||||
const html = `<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<meta name="color-scheme" content="dark light">
|
||||
<title>I.R.I.S. S3 API reference</title>
|
||||
</head>
|
||||
<body>
|
||||
<script id="api-reference" data-url="${OPENAPI_PATH}"></script>
|
||||
<script src="https://cdn.jsdelivr.net/npm/@scalar/api-reference"></script>
|
||||
</body>
|
||||
</html>`;
|
||||
|
||||
return new Response(html, {
|
||||
headers: {
|
||||
"Content-Type": "text/html; charset=utf-8",
|
||||
"Cache-Control": "public, max-age=300",
|
||||
},
|
||||
});
|
||||
}
|
||||
+2
-1
@@ -220,9 +220,10 @@ export async function streamDownloadFromDrive(accessToken: string, bucket: strin
|
||||
console.error(await downloadRes.text());
|
||||
throw new Error("Download failed");
|
||||
}
|
||||
if (!downloadRes.body) throw new Error("Download response had no body");
|
||||
|
||||
return {
|
||||
body: downloadRes.body!,
|
||||
body: downloadRes.body,
|
||||
contentType: file.mimeType || "application/octet-stream",
|
||||
size: parseInt(file.size || "0", 10),
|
||||
id: file.id,
|
||||
|
||||
+15
-7
@@ -1,5 +1,7 @@
|
||||
import { verifySignature } from "./aws-signature";
|
||||
import { isAllowedBucket, isPublicReadBucket } from "./bucket-access";
|
||||
import { preflightResponse, withCors } from "./cors";
|
||||
import * as docs from "./docs";
|
||||
import { getAccessToken } from "./google-drive";
|
||||
import { MultipartUploadDO } from "./multipart-do";
|
||||
import { dispatch } from "./router";
|
||||
@@ -10,27 +12,33 @@ export { MultipartUploadDO };
|
||||
|
||||
export default {
|
||||
async fetch(request: Request, env: Env, _ctx: ExecutionContext): Promise<Response> {
|
||||
if (request.method === "OPTIONS") return new Response(null, { status: 204 });
|
||||
if (request.method === "OPTIONS") return preflightResponse(request, env);
|
||||
|
||||
const url = new URL(request.url);
|
||||
if (env.ENABLE_DOCS !== "false") {
|
||||
if (request.method === "GET" && url.pathname === docs.OPENAPI_PATH) return withCors(docs.openApiResponse(), request, env);
|
||||
if (request.method === "GET" && url.pathname === docs.DOCS_PATH && !isAllowedBucket("docs", env)) return withCors(docs.docsResponse(), request, env);
|
||||
}
|
||||
|
||||
const pathParts = url.pathname.split("/").filter(Boolean);
|
||||
const bucket = pathParts[0] || "";
|
||||
const objectKey = pathParts.slice(1).join("/");
|
||||
const resource = url.pathname || "/";
|
||||
|
||||
try {
|
||||
if (!isAllowedBucket(bucket, env)) return s3Error("AccessDenied", 403, undefined, resource, request.method === "HEAD");
|
||||
if (!isAllowedBucket(bucket, env)) return withCors(s3Error("AccessDenied", 403, undefined, resource, request.method === "HEAD"), request, env);
|
||||
|
||||
const isPublicRead = isPublicReadBucket(bucket, env) && (request.method === "GET" || request.method === "HEAD");
|
||||
if (!isPublicRead && !(await verifySignature(request, env))) {
|
||||
return s3Error("SignatureDoesNotMatch", 403, undefined, resource, request.method === "HEAD");
|
||||
const signature = isPublicRead ? { ok: true as const } : await verifySignature(request, env);
|
||||
if (!signature.ok) {
|
||||
return withCors(s3Error(signature.code, 403, signature.message, resource, request.method === "HEAD"), request, env);
|
||||
}
|
||||
|
||||
return await dispatch(request, env, await getAccessToken(env), bucket, objectKey);
|
||||
return withCors(await dispatch(request, env, await getAccessToken(env), bucket, objectKey), request, env);
|
||||
} catch (error) {
|
||||
if (error instanceof S3Exception) return s3Error(error.code, error.status, error.message, resource, request.method === "HEAD", error.headers);
|
||||
if (error instanceof S3Exception) return withCors(s3Error(error.code, error.status, error.message, resource, request.method === "HEAD", error.headers), request, env);
|
||||
console.error(JSON.stringify({ message: "request failed", error: error instanceof Error ? error.message : String(error), method: request.method, path: url.pathname }));
|
||||
return s3Error("InternalError", 500, undefined, resource, request.method === "HEAD");
|
||||
return withCors(s3Error("InternalError", 500, undefined, resource, request.method === "HEAD"), request, env);
|
||||
}
|
||||
},
|
||||
} satisfies ExportedHandler<Env>;
|
||||
|
||||
+14
-8
@@ -57,6 +57,12 @@ export class MultipartUploadDO extends DurableObject<Env> {
|
||||
return this.ctx.storage.sql.exec<StateValueRow>("SELECT v FROM state WHERE k = ?", key).toArray()[0]?.v as T | undefined;
|
||||
}
|
||||
|
||||
private requiredValue<T extends ArrayBuffer | string | number>(key: string): T {
|
||||
const value = this.getValue<T>(key);
|
||||
if (value === undefined) throw new Error(`Missing multipart upload state: ${key}`);
|
||||
return value;
|
||||
}
|
||||
|
||||
private setValue(key: string, value: ArrayBuffer | string | number | null): void {
|
||||
this.ctx.storage.sql.exec("INSERT INTO state (k, v) VALUES (?, ?) ON CONFLICT(k) DO UPDATE SET v = excluded.v", key, value);
|
||||
}
|
||||
@@ -97,7 +103,7 @@ export class MultipartUploadDO extends DurableObject<Env> {
|
||||
private async resyncExpiredLease(inFlight: InFlight, requestId: string, partLen: number): Promise<BeginPartResult | null> {
|
||||
if (partLen !== inFlight.partLen) return { kind: "error", code: "InvalidPart", message: "A retried part must have the same decoded length" };
|
||||
const accessToken = await getAccessToken(this.env);
|
||||
const actualOffset = await queryStatus(this.getValue<string>("uploadUrl")!, accessToken);
|
||||
const actualOffset = await queryStatus(this.requiredValue<string>("uploadUrl"), accessToken);
|
||||
const partStartFileOffset = inFlight.driveOffsetAtStart + inFlight.carryLen;
|
||||
let skipBytes = 0;
|
||||
if (actualOffset === inFlight.driveOffsetAtStart) {
|
||||
@@ -129,7 +135,7 @@ export class MultipartUploadDO extends DurableObject<Env> {
|
||||
leaseExpiresAt: Date.now() + LEASE_MS,
|
||||
};
|
||||
this.setValue("inFlight", JSON.stringify(inFlight));
|
||||
return { kind: "admit", uploadUrl: this.getValue<string>("uploadUrl")!, driveOffset, carry, sendLen, skipBytes };
|
||||
return { kind: "admit", uploadUrl: this.requiredValue<string>("uploadUrl"), driveOffset, carry, sendLen, skipBytes };
|
||||
}
|
||||
|
||||
async beginPart(requestId: string, partNumber: number, partLen: number): Promise<BeginPartResult> {
|
||||
@@ -229,15 +235,15 @@ export class MultipartUploadDO extends DurableObject<Env> {
|
||||
const accessToken = await getAccessToken(this.env);
|
||||
let metadata: DriveUploadResult;
|
||||
if (total === 0) {
|
||||
await cancelSession(this.getValue<string>("uploadUrl")!, accessToken);
|
||||
await cancelSession(this.requiredValue<string>("uploadUrl"), accessToken);
|
||||
metadata = await createEmptyFile(accessToken, {
|
||||
name: this.getValue<string>("fileName")!,
|
||||
parents: [this.getValue<string>("parentFolderId")!],
|
||||
mimeType: this.getValue<string>("mimeType")!,
|
||||
name: this.requiredValue<string>("fileName"),
|
||||
parents: [this.requiredValue<string>("parentFolderId")],
|
||||
mimeType: this.requiredValue<string>("mimeType"),
|
||||
existingFileId: this.getValue<string>("existingFileId"),
|
||||
});
|
||||
} else {
|
||||
metadata = await putFinalChunk(this.getValue<string>("uploadUrl")!, accessToken, driveOffset, total, carry);
|
||||
metadata = await putFinalChunk(this.requiredValue<string>("uploadUrl"), accessToken, driveOffset, total, carry);
|
||||
}
|
||||
const partEtags = stored.map((part) => part.etag);
|
||||
await this.ctx.storage.deleteAlarm();
|
||||
@@ -259,7 +265,7 @@ export class MultipartUploadDO extends DurableObject<Env> {
|
||||
async abort(): Promise<boolean> {
|
||||
if (!this.hasUpload()) return false;
|
||||
const accessToken = await getAccessToken(this.env);
|
||||
await cancelSession(this.getValue<string>("uploadUrl")!, accessToken);
|
||||
await cancelSession(this.requiredValue<string>("uploadUrl"), accessToken);
|
||||
await this.ctx.storage.deleteAlarm();
|
||||
await this.ctx.storage.deleteAll();
|
||||
for (const waiter of this.waiters.values()) {
|
||||
|
||||
+2
-1
@@ -1,10 +1,11 @@
|
||||
import { escapeXml } from "./s3-xml";
|
||||
|
||||
export type S3ErrorCode = "AccessDenied" | "SignatureDoesNotMatch" | "NoSuchKey" | "NoSuchUpload" | "InvalidPart" | "InvalidPartOrder" | "EntityTooLarge" | "MalformedXML" | "InvalidArgument" | "MethodNotAllowed" | "NotImplemented" | "SlowDown" | "InternalError";
|
||||
export type S3ErrorCode = "AccessDenied" | "SignatureDoesNotMatch" | "RequestTimeTooSkewed" | "NoSuchKey" | "NoSuchUpload" | "InvalidPart" | "InvalidPartOrder" | "EntityTooLarge" | "MalformedXML" | "InvalidArgument" | "MethodNotAllowed" | "NotImplemented" | "SlowDown" | "InternalError";
|
||||
|
||||
const DEFAULT_MESSAGES: Record<S3ErrorCode, string> = {
|
||||
AccessDenied: "Access Denied",
|
||||
SignatureDoesNotMatch: "The request signature we calculated does not match the signature you provided.",
|
||||
RequestTimeTooSkewed: "The difference between the request time and the server's time is too large.",
|
||||
NoSuchKey: "The specified key does not exist.",
|
||||
NoSuchUpload: "The specified multipart upload does not exist.",
|
||||
InvalidPart: "One or more of the specified parts could not be found.",
|
||||
|
||||
Vendored
+4
@@ -0,0 +1,4 @@
|
||||
declare module "*.yaml" {
|
||||
const content: string;
|
||||
export default content;
|
||||
}
|
||||
@@ -12,6 +12,8 @@ export interface Env {
|
||||
PUBLIC_READ_BUCKETS?: string;
|
||||
ALLOW_MULTIPART?: string;
|
||||
ETAG_STYLE?: "md5" | "multipart";
|
||||
CORS_ALLOWED_ORIGINS?: string;
|
||||
ENABLE_DOCS?: string;
|
||||
}
|
||||
|
||||
export interface GoogleDriveFile {
|
||||
|
||||
@@ -0,0 +1,119 @@
|
||||
import { AwsClient } from "aws4fetch";
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
import { preflightResponse, withCors } from "../src/cors";
|
||||
import worker from "../src/index";
|
||||
import type { Env } from "../src/types";
|
||||
|
||||
import { env } from "cloudflare:test";
|
||||
|
||||
const ENV = env as unknown as Env;
|
||||
const ENDPOINT = "https://s3-api.example.com";
|
||||
const ORIGIN = "http://localhost:5173";
|
||||
const CTX = { waitUntil: vi.fn(), passThroughOnException: vi.fn() } as unknown as ExecutionContext;
|
||||
|
||||
async function signed(path: string, init: RequestInit): Promise<Request> {
|
||||
const aws = new AwsClient({ accessKeyId: ENV.ACCESS_KEY, secretAccessKey: ENV.SECRET_KEY, region: ENV.REGION, service: "s3" });
|
||||
const bodyLength = typeof init.body === "string" ? new TextEncoder().encode(init.body).byteLength : init.body instanceof Uint8Array ? init.body.byteLength : undefined;
|
||||
return aws.sign(`${ENDPOINT}${path}`, {
|
||||
...init,
|
||||
headers: { "x-amz-content-sha256": "UNSIGNED-PAYLOAD", ...(bodyLength === undefined ? {} : { "x-amz-decoded-content-length": String(bodyLength) }), ...init.headers },
|
||||
});
|
||||
}
|
||||
|
||||
function fakeGoogleFetch(input: string | URL | Request, init?: RequestInit): Promise<Response> {
|
||||
const request = input instanceof Request ? input : new Request(input, init);
|
||||
const url = new URL(request.url);
|
||||
if (url.hostname === "oauth2.googleapis.com") return Promise.resolve(Response.json({ access_token: "token", expires_in: 3600 }));
|
||||
if (url.pathname === "/drive/v3/files" && request.method === "GET") return Promise.resolve(Response.json({ files: [] }));
|
||||
if (url.pathname === "/drive/v3/files" && request.method === "POST") return Promise.resolve(Response.json({ id: "folder-1" }));
|
||||
if (url.pathname.startsWith("/upload/drive/v3/files")) return Promise.resolve(new Response(null, { headers: { Location: "https://www.googleapis.com/upload/session/test" } }));
|
||||
if (url.pathname === "/upload/session/test") return Promise.resolve(Response.json({ id: "file-1", name: "file.txt", md5Checksum: "d41d8cd98f00b204e9800998ecf8427e" }));
|
||||
return Promise.resolve(new Response("Not Found", { status: 404 }));
|
||||
}
|
||||
|
||||
afterEach(() => vi.unstubAllGlobals());
|
||||
|
||||
describe("CORS", () => {
|
||||
it("returns allow-list preflight headers", async () => {
|
||||
const response = preflightResponse(
|
||||
new Request(`${ENDPOINT}/test-bucket/file.txt`, {
|
||||
method: "OPTIONS",
|
||||
headers: { Origin: ORIGIN, "Access-Control-Request-Method": "PUT", "Access-Control-Request-Headers": "content-type,x-amz-date" },
|
||||
}),
|
||||
ENV,
|
||||
);
|
||||
|
||||
expect(response.status).toBe(204);
|
||||
expect(response.headers.get("Access-Control-Allow-Origin")).toBe(ORIGIN);
|
||||
expect(response.headers.get("Access-Control-Allow-Methods")).toBe("GET, HEAD, PUT, POST, DELETE, OPTIONS");
|
||||
expect(response.headers.get("Access-Control-Allow-Headers")).toBe("content-type,x-amz-date");
|
||||
expect(response.headers.get("Access-Control-Max-Age")).toBe("86400");
|
||||
expect(response.headers.get("Vary")).toBe("Origin");
|
||||
});
|
||||
|
||||
it("returns a bare 204 for a disallowed origin", async () => {
|
||||
const response = preflightResponse(new Request(`${ENDPOINT}/test-bucket/file.txt`, { method: "OPTIONS", headers: { Origin: "https://not-allowed.example", "Access-Control-Request-Method": "PUT" } }), ENV);
|
||||
|
||||
expect(response.status).toBe(204);
|
||||
expect([...response.headers]).toEqual([]);
|
||||
});
|
||||
|
||||
it("keeps OPTIONS requests without an Origin byte-compatible with the prior bare 204", () => {
|
||||
const response = preflightResponse(new Request(`${ENDPOINT}/test-bucket/file.txt`, { method: "OPTIONS" }), ENV);
|
||||
|
||||
expect(response.status).toBe(204);
|
||||
expect([...response.headers]).toEqual([]);
|
||||
});
|
||||
|
||||
it("exposes ETag for successful PUT responses", async () => {
|
||||
vi.stubGlobal("fetch", vi.fn(fakeGoogleFetch));
|
||||
const request = await signed("/test-bucket/file.txt", { method: "PUT", body: "hello", headers: { Origin: ORIGIN, "Content-Type": "text/plain" } });
|
||||
const response = await worker.fetch(request, ENV, CTX);
|
||||
|
||||
expect(response.status).toBe(200);
|
||||
expect(response.headers.get("ETag")).toBe('"d41d8cd98f00b204e9800998ecf8427e"');
|
||||
expect(response.headers.get("Access-Control-Allow-Origin")).toBe(ORIGIN);
|
||||
expect(response.headers.get("Access-Control-Expose-Headers")).toContain("ETag");
|
||||
});
|
||||
|
||||
it("adds CORS headers to access-denied and missing-key errors", async () => {
|
||||
const denied = await worker.fetch(new Request(`${ENDPOINT}/not-a-bucket`, { headers: { Origin: ORIGIN } }), ENV, CTX);
|
||||
expect(denied.status).toBe(403);
|
||||
expect(denied.headers.get("Access-Control-Allow-Origin")).toBe(ORIGIN);
|
||||
|
||||
vi.stubGlobal("fetch", vi.fn(fakeGoogleFetch));
|
||||
const missing = await worker.fetch(await signed("/test-bucket/missing.txt", { method: "GET", headers: { Origin: ORIGIN } }), ENV, CTX);
|
||||
expect(missing.status).toBe(404);
|
||||
expect(await missing.text()).toContain("<Code>NoSuchKey</Code>");
|
||||
expect(missing.headers.get("Access-Control-Allow-Origin")).toBe(ORIGIN);
|
||||
});
|
||||
|
||||
it("does not alter the S3 response for a disallowed Origin", async () => {
|
||||
const response = await worker.fetch(new Request(`${ENDPOINT}/not-a-bucket`, { headers: { Origin: "https://not-allowed.example" } }), ENV, CTX);
|
||||
|
||||
expect(response.status).toBe(403);
|
||||
expect(await response.text()).toContain("<Code>AccessDenied</Code>");
|
||||
expect(response.headers.get("Access-Control-Allow-Origin")).toBeNull();
|
||||
expect(response.headers.get("Access-Control-Expose-Headers")).toBeNull();
|
||||
expect(response.headers.get("Vary")).toBe("Origin");
|
||||
});
|
||||
|
||||
it("does not attach Access-Control headers to server-to-server responses without Origin", () => {
|
||||
const response = withCors(new Response("S3 response", { status: 200, headers: { ETag: '"etag"' } }), new Request(`${ENDPOINT}/test-bucket/file.txt`), ENV);
|
||||
|
||||
expect(response.status).toBe(200);
|
||||
expect(response.headers.get("ETag")).toBe('"etag"');
|
||||
expect(response.headers.get("Access-Control-Allow-Origin")).toBeNull();
|
||||
expect(response.headers.get("Access-Control-Expose-Headers")).toBeNull();
|
||||
expect(response.headers.get("Vary")).toBe("Origin");
|
||||
});
|
||||
|
||||
it("emits no CORS headers when CORS_ALLOWED_ORIGINS is unset", () => {
|
||||
const request = new Request(`${ENDPOINT}/test-bucket/file.txt`, { headers: { Origin: ORIGIN } });
|
||||
const response = withCors(new Response(null, { headers: { ETag: '"etag"' } }), request, { ...ENV, CORS_ALLOWED_ORIGINS: undefined });
|
||||
expect(response.headers.get("Access-Control-Allow-Origin")).toBeNull();
|
||||
expect(response.headers.get("Access-Control-Expose-Headers")).toBeNull();
|
||||
expect(response.headers.get("Vary")).toBeNull();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,39 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
|
||||
import worker from "../src/index";
|
||||
import type { Env } from "../src/types";
|
||||
|
||||
import { env } from "cloudflare:test";
|
||||
|
||||
const ENV = env as unknown as Env;
|
||||
const ENDPOINT = "https://s3-api.example.com";
|
||||
const CTX = { waitUntil: vi.fn(), passThroughOnException: vi.fn() } as unknown as ExecutionContext;
|
||||
|
||||
describe("API documentation routes", () => {
|
||||
it("serves the Scalar shell and raw OpenAPI document", async () => {
|
||||
const docs = await worker.fetch(new Request(`${ENDPOINT}/docs`), ENV, CTX);
|
||||
expect(docs.status).toBe(200);
|
||||
expect(docs.headers.get("Content-Type")).toContain("text/html");
|
||||
expect(await docs.text()).toContain("@scalar/api-reference");
|
||||
|
||||
const spec = await worker.fetch(new Request(`${ENDPOINT}/openapi.yaml`), ENV, CTX);
|
||||
expect(spec.status).toBe(200);
|
||||
expect(spec.headers.get("Content-Type")).toContain("application/yaml");
|
||||
expect(new TextDecoder().decode(await spec.arrayBuffer())).toContain("openapi: 3.1.0");
|
||||
});
|
||||
|
||||
it("bypasses documentation routes when disabled", async () => {
|
||||
const disabled = { ...ENV, ENABLE_DOCS: "false" };
|
||||
for (const path of ["/docs", "/openapi.yaml"]) {
|
||||
const response = await worker.fetch(new Request(`${ENDPOINT}${path}`), disabled, CTX);
|
||||
expect(response.status).toBe(403);
|
||||
}
|
||||
});
|
||||
|
||||
it("does not claim /docs when docs is a configured bucket", async () => {
|
||||
const withDocsBucket = { ...ENV, ALLOWED_BUCKETS: "test-bucket,docs" };
|
||||
const response = await worker.fetch(new Request(`${ENDPOINT}/docs`), withDocsBucket, CTX);
|
||||
expect(response.status).toBe(403);
|
||||
expect(await response.text()).toContain("<Code>SignatureDoesNotMatch</Code>");
|
||||
});
|
||||
});
|
||||
+64
-9
@@ -56,7 +56,9 @@ class FakeDrive {
|
||||
if (url.pathname === "/drive/v3/files" && request.method === "POST") return this.createMetadata(await request.json<Record<string, unknown>>());
|
||||
if (url.pathname.startsWith("/drive/v3/files/") && url.searchParams.get("alt") === "media") return this.download(url, request);
|
||||
if (url.pathname.startsWith("/drive/v3/files/") && request.method === "DELETE") {
|
||||
this.files.delete(url.pathname.split("/").at(-1)!);
|
||||
const fileId = url.pathname.split("/").at(-1);
|
||||
if (!fileId) return new Response("Not Found", { status: 404 });
|
||||
this.files.delete(fileId);
|
||||
return new Response(null, { status: 204 });
|
||||
}
|
||||
if (url.pathname.startsWith("/upload/drive/v3/files") && url.searchParams.get("uploadType") === "resumable") return this.initialize(url, request);
|
||||
@@ -121,7 +123,8 @@ class FakeDrive {
|
||||
}
|
||||
|
||||
private async upload(url: URL, request: Request): Promise<Response> {
|
||||
const id = url.pathname.split("/").at(-1)!;
|
||||
const id = url.pathname.split("/").at(-1);
|
||||
if (!id) return new Response(null, { status: 404 });
|
||||
const session = this.sessions.get(id);
|
||||
if (!session) return new Response(null, { status: 404 });
|
||||
if (request.method === "DELETE") {
|
||||
@@ -159,7 +162,9 @@ class FakeDrive {
|
||||
}
|
||||
|
||||
private download(url: URL, request: Request): Response {
|
||||
const file = this.files.get(url.pathname.split("/").at(-1)!);
|
||||
const fileId = url.pathname.split("/").at(-1);
|
||||
if (!fileId) return new Response(null, { status: 404 });
|
||||
const file = this.files.get(fileId);
|
||||
if (!file) return new Response(null, { status: 404 });
|
||||
const range = request.headers.get("Range");
|
||||
if (!range) return new Response(file.data, { headers: { "Content-Length": String(file.data.byteLength) } });
|
||||
@@ -204,6 +209,18 @@ async function signed(path: string, init: RequestInit): Promise<Request> {
|
||||
});
|
||||
}
|
||||
|
||||
async function presigned(path: string, init: RequestInit, options: { datetime?: string; accessKeyId?: string; expires?: string } = {}): Promise<Request> {
|
||||
const url = new URL(`${ENDPOINT}${path}`);
|
||||
url.searchParams.set("X-Amz-Expires", options.expires ?? "60");
|
||||
const aws = new AwsClient({ accessKeyId: options.accessKeyId ?? ENV.ACCESS_KEY, secretAccessKey: ENV.SECRET_KEY, region: ENV.REGION, service: "s3" });
|
||||
const bodyLength = typeof init.body === "string" ? new TextEncoder().encode(init.body).byteLength : init.body instanceof Uint8Array ? init.body.byteLength : undefined;
|
||||
return aws.sign(url.toString(), {
|
||||
...init,
|
||||
headers: { "x-amz-content-sha256": "UNSIGNED-PAYLOAD", ...(bodyLength === undefined ? {} : { "x-amz-decoded-content-length": String(bodyLength) }), ...init.headers },
|
||||
aws: { signQuery: true, ...(options.datetime ? { datetime: options.datetime } : {}) },
|
||||
});
|
||||
}
|
||||
|
||||
let drive: FakeDrive;
|
||||
|
||||
beforeEach(async () => {
|
||||
@@ -226,7 +243,10 @@ describe("S3 compatibility", () => {
|
||||
const second = await worker.fetch(await signed("/test-bucket/file.txt", { method: "PUT", body: "second" }), ENV, CTX);
|
||||
expect(second.status).toBe(200);
|
||||
expect([...drive.files.values()].filter((file) => file.name === "file.txt")).toHaveLength(1);
|
||||
expect(new TextDecoder().decode([...drive.files.values()].find((file) => file.name === "file.txt")!.data)).toBe("second");
|
||||
const stored = [...drive.files.values()].find((file) => file.name === "file.txt");
|
||||
expect(stored).toBeDefined();
|
||||
if (!stored) throw new Error("Overwritten object was not stored");
|
||||
expect(new TextDecoder().decode(stored.data)).toBe("second");
|
||||
});
|
||||
|
||||
it("sets Last-Modified on GET and HEAD from Drive's modifiedTime", async () => {
|
||||
@@ -264,6 +284,33 @@ describe("S3 compatibility", () => {
|
||||
expect(response.status).toBe(200);
|
||||
});
|
||||
|
||||
it("rejects expired presigned URLs and accepts unexpired ones", async () => {
|
||||
const expired = await worker.fetch(await presigned("/test-bucket/file.txt", { method: "GET" }, { datetime: "20200101T000000Z", expires: "60" }), ENV, CTX);
|
||||
expect(expired.status).toBe(403);
|
||||
expect(await expired.text()).toContain("<Code>AccessDenied</Code>");
|
||||
});
|
||||
|
||||
it("accepts an unexpired presigned URL", async () => {
|
||||
await worker.fetch(await signed("/test-bucket/presigned.txt", { method: "PUT", body: "hello" }), ENV, CTX);
|
||||
const response = await worker.fetch(await presigned("/test-bucket/presigned.txt", { method: "GET" }), ENV, CTX);
|
||||
expect(response.status).toBe(200);
|
||||
expect(await response.text()).toBe("hello");
|
||||
});
|
||||
|
||||
it("rejects a Credential access key that does not match ACCESS_KEY", async () => {
|
||||
const response = await worker.fetch(await presigned("/test-bucket/file.txt", { method: "GET" }, { accessKeyId: "unexpected-access-key" }), ENV, CTX);
|
||||
expect(response.status).toBe(403);
|
||||
expect(await response.text()).toContain("<Code>AccessDenied</Code>");
|
||||
});
|
||||
|
||||
it("rejects header-authenticated requests outside the 15-minute clock skew", async () => {
|
||||
const aws = new AwsClient({ accessKeyId: ENV.ACCESS_KEY, secretAccessKey: ENV.SECRET_KEY, region: ENV.REGION, service: "s3" });
|
||||
const request = await aws.sign(`${ENDPOINT}/test-bucket/file.txt`, { method: "GET", aws: { datetime: "20200101T000000Z" } });
|
||||
const response = await worker.fetch(request, ENV, CTX);
|
||||
expect(response.status).toBe(403);
|
||||
expect(await response.text()).toContain("<Code>RequestTimeTooSkewed</Code>");
|
||||
});
|
||||
|
||||
it("decodes both aws-chunked framing variants across arbitrary boundaries", async () => {
|
||||
const payload = bytes(70_013);
|
||||
for (const trailer of [true, false]) {
|
||||
@@ -282,20 +329,28 @@ describe("S3 compatibility", () => {
|
||||
const source = bytes(1_500_123);
|
||||
const create = await worker.fetch(await signed("/test-bucket/big.bin?uploads", { method: "POST", headers: { "Content-Type": "application/octet-stream" } }), ENV, CTX);
|
||||
expect(create.status).toBe(200);
|
||||
const uploadId = /<UploadId>([^<]+)<\/UploadId>/.exec(await create.text())![1];
|
||||
const uploadId = /<UploadId>([^<]+)<\/UploadId>/.exec(await create.text())?.[1];
|
||||
expect(uploadId).toBeDefined();
|
||||
if (!uploadId) throw new Error("Multipart initiation did not return an upload ID");
|
||||
const completed: Array<{ partNumber: number; etag: string }> = [];
|
||||
for (let index = 0, offset = 0; offset < source.byteLength; index++) {
|
||||
const end = Math.min(source.byteLength, offset + 500_000);
|
||||
const part = await worker.fetch(await signed(`/test-bucket/big.bin?partNumber=${index + 1}&uploadId=${encodeURIComponent(uploadId)}`, { method: "PUT", body: source.slice(offset, end) }), ENV, CTX);
|
||||
expect(part.status).toBe(200);
|
||||
completed.push({ partNumber: index + 1, etag: part.headers.get("ETag")!.replaceAll('"', "") });
|
||||
const partEtag = part.headers.get("ETag");
|
||||
expect(partEtag).toBeDefined();
|
||||
if (!partEtag) throw new Error(`Multipart part ${index + 1} did not return an ETag`);
|
||||
completed.push({ partNumber: index + 1, etag: partEtag.replaceAll('"', "") });
|
||||
offset = end;
|
||||
}
|
||||
const xml = `<CompleteMultipartUpload>${completed.map((part) => `<Part><PartNumber>${part.partNumber}</PartNumber><ETag>"${part.etag}"</ETag></Part>`).join("")}</CompleteMultipartUpload>`;
|
||||
const result = await worker.fetch(await signed(`/test-bucket/big.bin?uploadId=${encodeURIComponent(uploadId)}`, { method: "POST", body: xml }), ENV, CTX);
|
||||
expect(result.status).toBe(200);
|
||||
expect(await result.text()).toMatch(/<ETag>"[0-9a-f]{32}"<\/ETag>/);
|
||||
const stored = [...drive.files.values()].find((file) => file.name === "big.bin")!.data;
|
||||
const storedFile = [...drive.files.values()].find((file) => file.name === "big.bin");
|
||||
expect(storedFile).toBeDefined();
|
||||
if (!storedFile) throw new Error("Completed multipart object was not stored");
|
||||
const stored = storedFile.data;
|
||||
expect(stored.byteLength).toBe(source.byteLength);
|
||||
expect(fakeMd5(stored)).toBe(fakeMd5(source));
|
||||
});
|
||||
@@ -317,13 +372,13 @@ describe("S3 compatibility", () => {
|
||||
CTX,
|
||||
);
|
||||
expect(response.status).toBe(200);
|
||||
expect([...drive.files.values()].find((file) => file.name === "chunked.bin")!.data).toEqual(source);
|
||||
expect([...drive.files.values()].find((file) => file.name === "chunked.bin")?.data).toEqual(source);
|
||||
});
|
||||
|
||||
it("supports an empty PutObject", async () => {
|
||||
const response = await worker.fetch(await signed("/test-bucket/empty", { method: "PUT", body: new Uint8Array() }), ENV, CTX);
|
||||
expect(response.status).toBe(200);
|
||||
expect([...drive.files.values()].find((file) => file.name === "empty")!.data.byteLength).toBe(0);
|
||||
expect([...drive.files.values()].find((file) => file.name === "empty")?.data.byteLength).toBe(0);
|
||||
});
|
||||
|
||||
it("lists nested keys under a prefix, as CommonPrefixes with a delimiter and recursively without one", async () => {
|
||||
|
||||
@@ -16,6 +16,8 @@ export default defineConfig({
|
||||
ALLOWED_BUCKETS: "test-bucket,empty-bucket,my-bucket",
|
||||
ALLOW_MULTIPART: "true",
|
||||
ETAG_STYLE: "md5",
|
||||
CORS_ALLOWED_ORIGINS: "http://localhost:5173",
|
||||
ENABLE_DOCS: "true",
|
||||
},
|
||||
},
|
||||
}),
|
||||
|
||||
Vendored
+2272
-2750
File diff suppressed because it is too large
Load Diff
+53
-47
@@ -3,69 +3,75 @@
|
||||
* https://developers.cloudflare.com/workers/wrangler/configuration/
|
||||
*/
|
||||
{
|
||||
"$schema": "node_modules/wrangler/config-schema.json",
|
||||
"name": "iris",
|
||||
"main": "src/index.ts",
|
||||
"compatibility_date": "2025-09-27",
|
||||
"vars": {
|
||||
"ALLOW_MULTIPART": "true",
|
||||
"ETAG_STYLE": "md5"
|
||||
},
|
||||
"observability": {
|
||||
"enabled": true
|
||||
},
|
||||
"durable_objects": {
|
||||
"bindings": [
|
||||
{
|
||||
"name": "MPU",
|
||||
"class_name": "MultipartUploadDO"
|
||||
}
|
||||
]
|
||||
},
|
||||
"migrations": [
|
||||
{
|
||||
"tag": "v1",
|
||||
"new_sqlite_classes": [
|
||||
"MultipartUploadDO"
|
||||
]
|
||||
}
|
||||
],
|
||||
"kv_namespaces": [
|
||||
{
|
||||
"binding": "AUTH_KV",
|
||||
"id": "2113be46ce514e088572d0ef7459c1bf"
|
||||
},
|
||||
{
|
||||
"binding": "FOLDER_CACHE",
|
||||
"id": "6d2f9904a3ff4471b0f04025c1ae87cd"
|
||||
}
|
||||
]
|
||||
/**
|
||||
"$schema": "node_modules/wrangler/config-schema.json",
|
||||
"name": "iris",
|
||||
"main": "src/index.ts",
|
||||
"compatibility_date": "2025-09-27",
|
||||
"vars": {
|
||||
"ALLOW_MULTIPART": "true",
|
||||
"ETAG_STYLE": "md5",
|
||||
"CORS_ALLOWED_ORIGINS": "",
|
||||
"ENABLE_DOCS": "true"
|
||||
},
|
||||
"rules": [
|
||||
{
|
||||
"type": "Text",
|
||||
"globs": ["docs/openapi.yaml"]
|
||||
}
|
||||
],
|
||||
"observability": {
|
||||
"enabled": true
|
||||
},
|
||||
"durable_objects": {
|
||||
"bindings": [
|
||||
{
|
||||
"name": "MPU",
|
||||
"class_name": "MultipartUploadDO"
|
||||
}
|
||||
]
|
||||
},
|
||||
"migrations": [
|
||||
{
|
||||
"tag": "v1",
|
||||
"new_sqlite_classes": ["MultipartUploadDO"]
|
||||
}
|
||||
],
|
||||
"kv_namespaces": [
|
||||
{
|
||||
"binding": "AUTH_KV",
|
||||
"id": "2113be46ce514e088572d0ef7459c1bf"
|
||||
},
|
||||
{
|
||||
"binding": "FOLDER_CACHE",
|
||||
"id": "6d2f9904a3ff4471b0f04025c1ae87cd"
|
||||
}
|
||||
]
|
||||
/**
|
||||
* Smart Placement
|
||||
* https://developers.cloudflare.com/workers/configuration/smart-placement/#smart-placement
|
||||
*/
|
||||
// "placement": { "mode": "smart" }
|
||||
/**
|
||||
// "placement": { "mode": "smart" }
|
||||
/**
|
||||
* Bindings
|
||||
* Bindings allow your Worker to interact with resources on the Cloudflare Developer Platform, including
|
||||
* databases, object storage, AI inference, real-time communication and more.
|
||||
* https://developers.cloudflare.com/workers/runtime-apis/bindings/
|
||||
*/
|
||||
/**
|
||||
/**
|
||||
* Environment Variables
|
||||
* https://developers.cloudflare.com/workers/wrangler/configuration/#environment-variables
|
||||
* Note: Use secrets to store sensitive data.
|
||||
* https://developers.cloudflare.com/workers/configuration/secrets/
|
||||
*/
|
||||
// "vars": { "MY_VARIABLE": "production_value" }
|
||||
/**
|
||||
// "vars": { "MY_VARIABLE": "production_value" }
|
||||
/**
|
||||
* Static Assets
|
||||
* https://developers.cloudflare.com/workers/static-assets/binding/
|
||||
*/
|
||||
// "assets": { "directory": "./public/", "binding": "ASSETS" }
|
||||
/**
|
||||
// "assets": { "directory": "./public/", "binding": "ASSETS" }
|
||||
/**
|
||||
* Service Bindings (communicate between multiple Workers)
|
||||
* https://developers.cloudflare.com/workers/wrangler/configuration/#service-bindings
|
||||
*/
|
||||
// "services": [ { "binding": "MY_SERVICE", "service": "my-service" } ]
|
||||
}
|
||||
// "services": [ { "binding": "MY_SERVICE", "service": "my-service" } ]
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user