Upload Kmake

test/fixtures/wpt/common/dispatcher/README.md

@@ -0,0 +1,228 @@
+# `RemoteContext`: API for script execution in another context
+
+`RemoteContext` in `/common/dispatcher/dispatcher.js` provides an interface to
+execute JavaScript in another global object (page or worker, the "executor"),
+based on:
+
+- [WPT RFC 88: context IDs from uuid searchParams in URL](https://github.com/web-platform-tests/rfcs/pull/88),
+- [WPT RFC 89: execute_script](https://github.com/web-platform-tests/rfcs/pull/89) and
+- [WPT RFC 91: RemoteContext](https://github.com/web-platform-tests/rfcs/pull/91).
+
+Tests can send arbitrary javascript to executors to evaluate in its global
+object, like:
+
+```
+// injector.html
+const argOnLocalContext = ...;
+
+async function execute() {
+  window.open('executor.html?uuid=' + uuid);
+  const ctx = new RemoteContext(uuid);
+  await ctx.execute_script(
+      (arg) => functionOnRemoteContext(arg),
+      [argOnLocalContext]);
+};
+```
+
+and on executor:
+
+```
+// executor.html
+function functionOnRemoteContext(arg) { ... }
+
+const uuid = new URLSearchParams(window.location.search).get('uuid');
+const executor = new Executor(uuid);
+```
+
+For concrete examples, see
+[events.html](../../html/browsers/browsing-the-web/back-forward-cache/events.html)
+and
+[executor.html](../../html/browsers/browsing-the-web/back-forward-cache/resources/executor.html)
+in back-forward cache tests.
+
+Note that `executor*` files under `/common/dispatcher/` are NOT for
+`RemoteContext.execute_script()`. Use `remote-executor.html` instead.
+
+This is universal and avoids introducing many specific `XXX-helper.html`
+resources.
+Moreover, tests are easier to read, because the whole logic of the test can be
+defined in a single file.
+
+## `new RemoteContext(uuid)`
+
+- `uuid` is a UUID string that identifies the remote context and should match
+  with the `uuid` parameter of the URL of the remote context.
+- Callers should create the remote context outside this constructor (e.g.
+  `window.open('executor.html?uuid=' + uuid)`).
+
+## `RemoteContext.execute_script(fn, args)`
+
+- `fn` is a JavaScript function to execute on the remote context, which is
+  converted to a string using `toString()` and sent to the remote context.
+- `args` is null or an array of arguments to pass to the function on the
+  remote context. Arguments are passed as JSON.
+- If the return value of `fn` when executed in the remote context is a promise,
+  the promise returned by `execute_script` resolves to the resolved value of
+  that promise. Otherwise the `execute_script` promise resolves to the return
+  value of `fn`.
+
+Note that `fn` is evaluated on the remote context (`executor.html` in the
+example above), while `args` are evaluated on the caller context
+(`injector.html`) and then passed to the remote context.
+
+## Return value of injected functions and `execute_script()`
+
+If the return value of the injected function when executed in the remote
+context is a promise, the promise returned by `execute_script` resolves to the
+resolved value of that promise. Otherwise the `execute_script` promise resolves
+to the return value of the function.
+
+When the return value of an injected script is a Promise, it should be resolved
+before any navigation starts on the remote context. For example, it shouldn't
+be resolved after navigating out and navigating back to the page again.
+It's fine to create a Promise to be resolved after navigations, if it's not the
+return value of the injected function.
+
+## Calling timing of `execute_script()`
+
+When `RemoteContext.execute_script()` is called when the remote context is not
+active (for example before it is created, before navigation to the page, or
+during the page is in back-forward cache), the injected script is evaluated
+after the remote context becomes active.
+
+Multiple calls to `RemoteContext.execute_script()` will result in multiple scripts
+being executed in remote context and ordering will be maintained.
+
+## Errors from `execute_script()`
+
+Errors from `execute_script()` will result in promise rejections, so it is
+important to await the result.  This can be `await ctx.execute_script(...)` for
+every call but if there are multiple scripts to executed, it may be preferable
+to wait on them in parallel to avoid incurring full round-trip time for each,
+e.g.
+
+```js
+await Promise.all(
+  ctx1.execute_script(...),
+  ctx1.execute_script(...),
+  ctx2.execute_script(...),
+  ctx2.execute_script(...),
+  ...
+)
+```
+
+## Evaluation timing of injected functions
+
+The script injected by `RemoteContext.execute_script()` can be evaluated any
+time during the remote context is active.
+For example, even before DOMContentLoaded events or even during navigation.
+It's the responsibility of test-specific code/helpers to ensure evaluation
+timing constraints (which can be also test-specific), if any needed.
+
+### Ensuring evaluation timing around page load
+
+For example, to ensure that injected functions (`mainFunction` below) are
+evaluated after the first `pageshow` event, we can use pure JavaScript code
+like below:
+
+```
+// executor.html
+window.pageShowPromise = new Promise(resolve =>
+    window.addEventListener('pageshow', resolve, {once: true}));
+
+
+// injector.html
+const waitForPageShow = async () => {
+  while (!window.pageShowPromise) {
+    await new Promise(resolve => setTimeout(resolve, 100));
+  }
+  await window.pageShowPromise;
+};
+
+await ctx.execute(waitForPageShow);
+await ctx.execute(mainFunction);
+```
+
+### Ensuring evaluation timing around navigation out/unloading
+
+It can be important to ensure there are no injected functions nor code behind
+`RemoteContext` (such as Fetch APIs accessing server-side stash) running after
+navigation is initiated, for example in the case of back-forward cache testing.
+
+To ensure this,
+
+- Do not call the next `RemoteContext.execute()` for the remote context after
+  triggering the navigation, until we are sure that the remote context is not
+  active (e.g. after we confirm that the new page is loaded).
+- Call `Executor.suspend(callback)` synchronously within the injected script.
+  This suspends executor-related code, and calls `callback` when it is ready
+  to start navigation.
+
+The code on the injector side would be like:
+
+```
+// injector.html
+await ctx.execute_script(() => {
+  executor.suspend(() => {
+    location.href = 'new-url.html';
+  });
+});
+```
+
+## Future Work: Possible integration with `test_driver`
+
+Currently `RemoteContext` is implemented by JavaScript and WPT-server-side
+stash, and not integrated with `test_driver` nor `testharness`.
+There is a proposal of `test_driver`-integrated version (see the RFCs listed
+above).
+
+The API semantics and guidelines in this document are designed to be applicable
+to both the current stash-based `RemoteContext` and `test_driver`-based
+version, and thus the tests using `RemoteContext` will be migrated with minimum
+modifications (mostly in `/common/dispatcher/dispatcher.js` and executors), for
+example in a
+[draft CL](https://chromium-review.googlesource.com/c/chromium/src/+/3082215/).
+
+
+# `send()`/`receive()` Message passing APIs
+
+`dispatcher.js` (and its server-side backend `dispatcher.py`) provides a
+universal queue-based message passing API.
+Each queue is identified by a UUID, and accessed via the following APIs:
+
+-   `send(uuid, message)` pushes a string `message` to the queue `uuid`.
+-   `receive(uuid)` pops the first item from the queue `uuid`.
+-   `showRequestHeaders(origin, uuid)` and
+    `cacheableShowRequestHeaders(origin, uuid)` return URLs, that push request
+    headers to the queue `uuid` upon fetching.
+
+It works cross-origin, and even access different browser context groups.
+
+Messages are queued, this means one doesn't need to wait for the receiver to
+listen, before sending the first message
+(but still need to wait for the resolution of the promise returned by `send()`
+to ensure the order between `send()`s).
+
+## Executors
+
+Similar to `RemoteContext.execute_script()`, `send()`/`receive()` can be used
+for sending arbitrary javascript to be evaluated in another page or worker.
+
+- `executor.html` (as a Document),
+- `executor-worker.js` (as a Web Worker), and
+- `executor-service-worker.js` (as a Service Worker)
+
+are examples of executors.
+Note that these executors are NOT compatible with
+`RemoteContext.execute_script()`.
+
+## Future Work
+
+`send()`, `receive()` and the executors below are kept for COEP/COOP tests.
+
+For remote script execution, new tests should use
+`RemoteContext.execute_script()` instead.
+
+For message passing,
+[WPT RFC 90](https://github.com/web-platform-tests/rfcs/pull/90) is still under
+discussion.

test/fixtures/wpt/common/dispatcher/dispatcher.js

@@ -0,0 +1,256 @@
+// Define a universal message passing API. It works cross-origin and across
+// browsing context groups.
+const dispatcher_path = "/common/dispatcher/dispatcher.py";
+const dispatcher_url = new URL(dispatcher_path, location.href).href;
+
+// Return a promise, limiting the number of concurrent accesses to a shared
+// resources to |max_concurrent_access|.
+const concurrencyLimiter = (max_concurrency) => {
+  let pending = 0;
+  let waiting = [];
+  return async (task) => {
+    pending++;
+    if (pending > max_concurrency)
+      await new Promise(resolve => waiting.push(resolve));
+    let result = await task();
+    pending--;
+    waiting.shift()?.();
+    return result;
+  };
+}
+
+// Wait for a random amount of time in the range [10ms,100ms].
+const randomDelay = () => {
+  return new Promise(resolve => setTimeout(resolve, 10 + 90*Math.random()));
+}
+
+// Sending too many requests in parallel causes congestion. Limiting it improves
+// throughput.
+//
+// Note: The following table has been determined on the test:
+// ../cache-storage.tentative.https.html
+// using Chrome with a 64 core CPU / 64GB ram, in release mode:
+// ┌───────────┬───┬───┬───┬───┬───┬───┬───┬───┬───┬───┬───┬────┐
+// │concurrency│ 1 │ 2 │ 3 │ 4 │ 5 │ 6 │ 10│ 15│ 20│ 30│ 50│ 100│
+// ├───────────┼───┼───┼───┼───┼───┼───┼───┼───┼───┼───┼───┼────┤
+// │time (s)   │ 54│ 38│ 31│ 29│ 26│ 24│ 22│ 22│ 22│ 22│ 34│ 36 │
+// └───────────┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴────┘
+const limiter = concurrencyLimiter(6);
+
+// While requests to different remote contexts can go in parallel, we need to
+// ensure that requests to each remote context are done in order. This maps a
+// uuid to a queue of requests to send. A queue is processed until it is empty
+// and then is deleted from the map.
+const sendQueues = new Map();
+
+// Sends a single item (with rate-limiting) and calls the associated resolver
+// when it is successfully sent.
+const sendItem = async function (uuid, resolver, message) {
+  await limiter(async () => {
+    // Requests might be dropped. Retry until getting a confirmation it has been
+    // processed.
+    while(1) {
+      try {
+        let response = await fetch(dispatcher_url + `?uuid=${uuid}`, {
+          method: 'POST',
+          body: message
+        })
+        if (await response.text() == "done") {
+          resolver();
+          return;
+        }
+      } catch (fetch_error) {}
+      await randomDelay();
+    };
+  });
+}
+
+// While the queue is non-empty, send the next item. This is async and new items
+// may be added to the queue while others are being sent.
+const processQueue = async function (uuid, queue) {
+  while (queue.length) {
+    const [resolver, message] = queue.shift();
+    await sendItem(uuid, resolver, message);
+  }
+  // The queue is empty, delete it.
+  sendQueues.delete(uuid);
+}
+
+const send = async function (uuid, message) {
+  const itemSentPromise = new Promise((resolve) => {
+    const item = [resolve, message];
+    if (sendQueues.has(uuid)) {
+      // There is already a queue for `uuid`, just add to it and it will be processed.
+      sendQueues.get(uuid).push(item);
+    } else {
+      // There is no queue for `uuid`, create it and start processing.
+      const queue = [item];
+      sendQueues.set(uuid, queue);
+      processQueue(uuid, queue);
+    }
+  });
+  // Wait until the item has been successfully sent.
+  await itemSentPromise;
+}
+
+const receive = async function (uuid) {
+  while(1) {
+    let data = "not ready";
+    try {
+      data = await limiter(async () => {
+        let response = await fetch(dispatcher_url + `?uuid=${uuid}`);
+        return await response.text();
+      });
+    } catch (fetch_error) {}
+
+    if (data == "not ready") {
+      await randomDelay();
+      continue;
+    }
+
+    return data;
+  }
+}
+
+// Returns an URL. When called, the server sends toward the `uuid` queue the
+// request headers. Useful for determining if something was requested with
+// Cookies.
+const showRequestHeaders = function(origin, uuid) {
+  return origin + dispatcher_path + `?uuid=${uuid}&show-headers`;
+}
+
+// Same as above, except for the response is cacheable.
+const cacheableShowRequestHeaders = function(origin, uuid) {
+  return origin + dispatcher_path + `?uuid=${uuid}&cacheable&show-headers`;
+}
+
+// This script requires
+// - `/common/utils.js` for `token()`.
+
+// Returns the URL of a document that can be used as a `RemoteContext`.
+//
+// `uuid` should be a UUID uniquely identifying the given remote context.
+// `options` has the following shape:
+//
+// {
+//   host: (optional) Sets the returned URL's `host` property. Useful for
+//     cross-origin executors.
+//   protocol: (optional) Sets the returned URL's `protocol` property.
+// }
+function remoteExecutorUrl(uuid, options) {
+  const url = new URL("/common/dispatcher/remote-executor.html", location);
+  url.searchParams.set("uuid", uuid);
+
+  if (options?.host) {
+    url.host = options.host;
+  }
+
+  if (options?.protocol) {
+    url.protocol = options.protocol;
+  }
+
+  return url;
+}
+
+// Represents a remote executor. For more detailed explanation see `README.md`.
+class RemoteContext {
+  // `uuid` is a UUID string that identifies the remote context and should
+  // match with the `uuid` parameter of the URL of the remote context.
+  constructor(uuid) {
+    this.context_id = uuid;
+  }
+
+  // Evaluates the script `expr` on the executor.
+  // - If `expr` is evaluated to a Promise that is resolved with a value:
+  //   `execute_script()` returns a Promise resolved with the value.
+  // - If `expr` is evaluated to a non-Promise value:
+  //   `execute_script()` returns a Promise resolved with the value.
+  // - If `expr` throws an error or is evaluated to a Promise that is rejected:
+  //   `execute_script()` returns a rejected Promise with the error's
+  //   `message`.
+  //   Note that currently the type of error (e.g. DOMException) is not
+  //   preserved, except for `TypeError`.
+  // The values should be able to be serialized by JSON.stringify().
+  async execute_script(fn, args) {
+    const receiver = token();
+    await this.send({receiver: receiver, fn: fn.toString(), args: args});
+    const response = JSON.parse(await receive(receiver));
+    if (response.status === 'success') {
+      return response.value;
+    }
+
+    // exception
+    if (response.name === 'TypeError') {
+      throw new TypeError(response.value);
+    }
+    throw new Error(response.value);
+  }
+
+  async send(msg) {
+    return await send(this.context_id, JSON.stringify(msg));
+  }
+};
+
+class Executor {
+  constructor(uuid) {
+    this.uuid = uuid;
+
+    // If `suspend_callback` is not `null`, the executor should be suspended
+    // when there are no ongoing tasks.
+    this.suspend_callback = null;
+
+    this.execute();
+  }
+
+  // Wait until there are no ongoing tasks nor fetch requests for polling
+  // tasks, and then suspend the executor and call `callback()`.
+  // Navigation from the executor page should be triggered inside `callback()`,
+  // to avoid conflict with in-flight fetch requests.
+  suspend(callback) {
+    this.suspend_callback = callback;
+  }
+
+  resume() {
+  }
+
+  async execute() {
+    while(true) {
+      if (this.suspend_callback !== null) {
+        this.suspend_callback();
+        this.suspend_callback = null;
+        // Wait for `resume()` to be called.
+        await new Promise(resolve => this.resume = resolve);
+
+        // Workaround for https://crbug.com/1244230.
+        // Without this workaround, the executor is resumed and the fetch
+        // request to poll the next task is initiated synchronously from
+        // pageshow event after the page restored from BFCache, and the fetch
+        // request promise is never resolved (and thus the test results in
+        // timeout) due to https://crbug.com/1244230. The root cause is not yet
+        // known, but setTimeout() with 0ms causes the resume triggered on
+        // another task and seems to resolve the issue.
+        await new Promise(resolve => setTimeout(resolve, 0));
+
+        continue;
+      }
+
+      const task = JSON.parse(await receive(this.uuid));
+
+      let response;
+      try {
+        const value = await eval(task.fn).apply(null, task.args);
+        response = JSON.stringify({
+          status: 'success',
+          value: value
+        });
+      } catch(e) {
+        response = JSON.stringify({
+          status: 'exception',
+          name: e.name,
+          value: e.message
+        });
+      }
+      await send(task.receiver, response);
+    }
+  }
+}

test/fixtures/wpt/common/dispatcher/executor-service-worker.js

@@ -0,0 +1,24 @@
+importScripts('./dispatcher.js');
+
+const params = new URLSearchParams(location.search);
+const uuid = params.get('uuid');
+
+// The fetch handler must be registered before parsing the main script response.
+// So do it here, for future use.
+fetchHandler = () => {}
+addEventListener('fetch', e => {
+  fetchHandler(e);
+});
+
+// Force ServiceWorker to immediately activate itself.
+addEventListener('install', event => {
+  skipWaiting();
+});
+
+let executeOrders = async function() {
+  while(true) {
+    let task = await receive(uuid);
+    eval(`(async () => {${task}})()`);
+  }
+};
+executeOrders();

test/fixtures/wpt/common/dispatcher/executor-worker.js

@@ -0,0 +1,12 @@
+importScripts('./dispatcher.js');
+
+const params = new URLSearchParams(location.search);
+const uuid = params.get('uuid');
+
+let executeOrders = async function() {
+  while(true) {
+    let task = await receive(uuid);
+    eval(`(async () => {${task}})()`);
+  }
+};
+executeOrders();

test/fixtures/wpt/common/dispatcher/executor.html

@@ -0,0 +1,15 @@
+<script src="./dispatcher.js"></script>
+<script>
+
+const params = new URLSearchParams(window.location.search);
+const uuid = params.get('uuid');
+
+let executeOrders = async function() {
+  while(true) {
+    let task = await receive(uuid);
+    eval(`(async () => {${task}})()`);
+  }
+};
+executeOrders();
+
+</script>

test/fixtures/wpt/common/dispatcher/remote-executor.html

@@ -0,0 +1,12 @@
+<!DOCTYPE html>
+<html>
+<meta charset="utf-8">
+<body>
+</body>
+<script src="./dispatcher.js"></script>
+<script>
+  const params = new URLSearchParams(window.location.search);
+  const uuid = params.get('uuid');
+  const executor = new Executor(uuid);  // `execute()` is called in constructor.
+</script>
+</html>