Upload Kmake

This commit is contained in:
Gorochu
2026-05-26 23:36:42 -07:00
parent ba051b2f74
commit 555ec72358
41615 changed files with 13344630 additions and 1 deletions

View File

@ -0,0 +1,52 @@
# Contributing a new API to Node-API
Node-API is the ABI-stable API for native addons. We encourage contributions to enhance the API,
while also ensuring compatibility and adherence to guidelines. When adding a new API to Node-API,
please follow these principles and guidelines:
## Core principles
1. **Adherence to Node-API standards**
* **Must** be a C API.
* **Must** not throw exceptions.
* **Must** return `napi_status`.
* **Should** consume `napi_env`.
* **Must** operate on primitive data types, pointers to primitive data types, or opaque handles.
* **Must** be a necessary API, not a convenience API (which belongs in node-addon-api).
* **Must** not break ABI compatibility with other Node.js versions.
2. **Maintaining VM agnosticism**
* New APIs **should** be compatible with various JavaScript VMs.
## Documentation and testing
1. **Documentation**
* PRs introducing new APIs **must** include corresponding documentation updates.
* Experimental APIs **must** be clearly documented as experimental and require an explicit compile-time flag
to opt-in (`#define`).
2. **Testing**
* PRs **must** include at least one test case demonstrating API usage.
* **Should** include test cases for various interesting uses of the API.
* **Should** provide a sample demonstrating realistic usage, similar to a real-world addon.
## Process and approval
1. **Team discussion**
* New APIs **should** be discussed in a Node-API team meeting.
2. **Review and approval**
* A new API addition **must** be signed off by at least two Node-API team members.
* **Should** be implemented in at least one other VM implementation of Node.js.
3. **Experimental phase**
* New APIs **must** be marked as experimental for at least one minor Node.js release before promotion.
* **Must** have a feature flag (`NODE_API_EXPERIMENTAL_HAS_<FEATURE>`) for distinguishing experimental
feature existence.
* **Must** be considered for backporting.
* Exit criteria from experimental status include:
* Opening a PR in `nodejs/node` to remove experimental status, tagged as **node-api** and **semver-minor**.
* Approval by the Node-API team.
* Availability of a down-level implementation if backporting is needed.
* Usage by a published real-world module.
* Implementation in an alternative VM.

View File

@ -0,0 +1,214 @@
# Adding V8 Fast API
Node.js uses [V8](https://v8.dev/) as its JavaScript engine.
Embedding functions implemented in C++ incur a high overhead, so V8
provides an API to implement native functions which may be invoked directly
from JIT-ed code. These functions also come with additional constraints,
for example, they may not trigger garbage collection.
## Limitations
* Fast API functions may not trigger garbage collection. This means by proxy
that JavaScript execution and heap allocation are also forbidden, including
`v8::Array::Get()` or `v8::Number::New()`.
* Throwing errors is not available from within a fast API call, but can be done
through the fallback to the slow API.
* Not all parameter and return types are supported in fast API calls.
For a full list, please look into
[`v8-fast-api-calls.h`](../../deps/v8/include/v8-fast-api-calls.h).
## Requirements
* Any function passed to `CFunction::Make`, including fast API function
declarations, should have their signature registered in
[`node_external_reference.h`](../../src/node_external_reference.h) file.
Although, it would not start failing or crashing until the function ends up
in a snapshot (either the built-in or a user-land one). Please refer to the
[binding functions documentation](../../src/README.md#binding-functions) for more
information.
* To test fast APIs, make sure to run the tests in a loop with a decent
iterations count to trigger relevant optimizations that prefer the fast API
over the slow one.
* In debug mode (`--debug` or `--debug-node` flags), the fast API calls can be
tracked using the `TRACK_V8_FAST_API_CALL("key")` macro. This can be used to
count how many times fast paths are taken during tests. The key is a global
identifier and should be unique across the codebase.
Use `"binding_name.function_name"` or `"binding_name.function_name.suffix"` to
ensure uniqueness.
* The fast callback must be idempotent up to the point where error and fallback
conditions are checked, because otherwise executing the slow callback might
produce visible side effects twice.
* If the receiver is used in the callback, it must be passed as a second argument,
leaving the first one unused, to prevent the JS land from accidentally omitting the receiver when
invoking the fast API method.
```cpp
// Instead of invoking the method as `receiver.internalModuleStat(input)`, the JS land should
// invoke it as `internalModuleStat(binding, input)` to make sure the binding is available to
// the native land.
static int32_t FastInternalModuleStat(
Local<Object> unused,
Local<Object> recv,
const FastOneByteString& input,
FastApiCallbackOptions& options) {
Environment* env = Environment::GetCurrent(recv->GetCreationContextChecked());
// More code
}
```
## Fallback to slow path
Fast API supports fallback to slow path for when it is desirable to do so,
for example, when throwing a custom error or executing JavaScript code is
needed. The fallback mechanism can be enabled and changed from the C++
implementation of the fast API function declaration.
Passing `true` to the `fallback` option will force V8 to run the slow path
with the same arguments.
In V8, the options fallback is defined as `FastApiCallbackOptions` inside
[`v8-fast-api-calls.h`](../../deps/v8/include/v8-fast-api-calls.h).
* C++ land
Example of a conditional fast path on C++
```cpp
// Anywhere in the execution flow, you can set fallback and stop the execution.
static double divide(const int32_t a,
const int32_t b,
v8::FastApiCallbackOptions& options) {
if (b == 0) {
options.fallback = true;
return 0;
} else {
return a / b;
}
}
```
## Example
A typical function that communicates between JavaScript and C++ is as follows.
* On the JavaScript side:
```js
const { divide } = internalBinding('custom_namespace');
```
* On the C++ side:
```cpp
#include "node_debug.h"
#include "v8-fast-api-calls.h"
namespace node {
namespace custom_namespace {
static void SlowDivide(const FunctionCallbackInfo<Value>& args) {
Environment* env = Environment::GetCurrent(args);
CHECK_GE(args.Length(), 2);
CHECK(args[0]->IsInt32());
CHECK(args[1]->IsInt32());
auto a = args[0].As<v8::Int32>();
auto b = args[1].As<v8::Int32>();
if (b->Value() == 0) {
return node::THROW_ERR_INVALID_STATE(env, "Error");
}
double result = a->Value() / b->Value();
args.GetReturnValue().Set(v8::Number::New(env->isolate(), result));
}
static double FastDivide(const int32_t a,
const int32_t b,
v8::FastApiCallbackOptions& options) {
if (b == 0) {
TRACK_V8_FAST_API_CALL("custom_namespace.divide.error");
options.fallback = true;
return 0;
} else {
TRACK_V8_FAST_API_CALL("custom_namespace.divide.ok");
return a / b;
}
}
CFunction fast_divide_(CFunction::Make(FastDivide));
static void Initialize(Local<Object> target,
Local<Value> unused,
Local<Context> context,
void* priv) {
SetFastMethod(context, target, "divide", SlowDivide, &fast_divide_);
}
void RegisterExternalReferences(ExternalReferenceRegistry* registry) {
registry->Register(SlowDivide);
registry->Register(FastDivide);
registry->Register(fast_divide_.GetTypeInfo());
}
} // namespace custom_namespace
} // namespace node
NODE_BINDING_CONTEXT_AWARE_INTERNAL(custom_namespace,
node::custom_namespace::Initialize);
NODE_BINDING_EXTERNAL_REFERENCE(
custom_namespace,
node::custom_namespace::RegisterExternalReferences);
```
* Update external references ([`node_external_reference.h`](../../src/node_external_reference.h))
Since our implementation used
`double(const int32_t a, const int32_t b, v8::FastApiCallbackOptions& options)`
signature, we need to add it to external references and in
`ALLOWED_EXTERNAL_REFERENCE_TYPES`.
Example declaration:
```cpp
using CFunctionCallbackReturningDouble = double (*)(const int32_t a,
const int32_t b,
v8::FastApiCallbackOptions& options);
```
* In the unit tests:
Since the fast API function uses `TRACK_V8_FAST_API_CALL`, we can ensure that
the fast paths are taken and test them by writing tests that force
V8 optimizations and check the counters.
```js
// Flags: --expose-internals --no-warnings --allow-natives-syntax
'use strict';
const common = require('../common');
const { internalBinding } = require('internal/test/binding');
// We could also require a function that uses the internal binding internally.
const { divide } = internalBinding('custom_namespace');
// The function that will be optimized. It has to be a function written in
// JavaScript. Since `divide` comes from the C++ side, we need to wrap it.
function testFastPath(a, b) {
return divide(a, b);
}
eval('%PrepareFunctionForOptimization(testFastPath)');
// This call will let V8 know about the argument types that the function expects.
assert.strictEqual(testFastPath(6, 3), 2);
eval('%OptimizeFunctionOnNextCall(testFastPath)');
assert.strictEqual(testFastPath(8, 2), 4);
assert.throws(() => testFastPath(1, 0), {
code: 'ERR_INVALID_STATE',
});
if (common.isDebug) {
const { getV8FastApiCallCount } = internalBinding('debug');
assert.strictEqual(getV8FastApiCallCount('custom_namespace.divide.ok'), 1);
assert.strictEqual(getV8FastApiCallCount('custom_namespace.divide.error'), 1);
}
```

View File

@ -0,0 +1,293 @@
# Advocacy Ambassador program
Node.js aims to support and encourage developers and developer
advocates to promote Node.js as a project and to help communicate
key messages to their constituencies.
The ambassador program does that by:
* Maintaining an up-to-date set of messages that the Node.js maintainers would like ambassadors
to promote. These will be documented in the nodejs/ambassadors repository.
* Maintaining an up-to-date set of topics on which additional content
would benefit the community. These will be documented in the
nodejs/ambassadors repository.
* Providing a space for collaborators to share new topics/features they
are working on when they would be happy for ambassadors to help share and get
early feedback.
* Helping ambassadors promote content that aligns with the
messages and topics defined.
* Advocating for ambassadors to be part of the OpenJS speakers bureau, even if the
ambassador is not otherwise an active member of the project itself.
* Each ambassador could add a maximum of three links to resources to learn Node.js
on a dedicated page on the main Node.js website. At least one of those must be a
free resource. The Node.js TSC members could ask for coupon codes to verify the
material if they so decide.
## Ambassadors nominations
Any member of the Node.js organization may nominate an ambassador.
The Technical Steering Committee (TSC) will review/approve ambassadors.
In the initial phase of the program:
* There will be a maximum of 4 ambassadors.
* The term will be for the calendar year. After a year, ambassadors automatically
drop out of the ambassador list and must be re-nominated.
* Ambassadors will be documented on the nodejs/ambassadors `README.md`.
The TSC may remove an ambassador for any reason,
including promoting content misaligned with the project.
Members of the Node.js organization can nominate ambassadors by opening
a discussion issue with the nomination in
[nodejs/collaborators](https://github.com/nodejs/collaborators/)
titled `Nomination X to be an Ambassador`, where X is the name of the person
being nominated.
If there is no objection within 14 days, the nomination is approved.
## Onboarding
To onboard an ambassador, a member of the TSC will:
* \[ ] Add the ambassador to the nodejs/ambassadors team.
* \[ ] Add the ambassador to the nodejs/ambassadors `README.md`.
* \[ ] Add the ambassador to the OpenJS Slack channel.
## Reviewing content
Ambassadors are asked to share links to new content before asking
for the content to be promoted. The goal is to provide the opportunity
for collaborators to help improve content as opposed to a detailed
nitpick review.
This is done by opening an issue in the nodejs/ambassadors
repository, which limits access to members of the Node.js
organization. The issue may include the link to existing published
content or a draft of the content itself.
Interested members of the Node.js organization may comment
in the issue and the ambassadors are expected to
consider those suggestions and update the content when they feel it
is appropriate.
Explicit approval is not required in the issue (i.e. if there is no
input, that's OK), but review comments should be addressed before
requesting promotion.
## Expectations on Ambassadors
Ambassadors are expected to engage both within the project and externally
in a manner that is consistent with the project's
[Code of Conduct](https://github.com/nodejs/admin/blob/main/CODE_OF_CONDUCT.md)
as well as the additional expectations outlined in
[Member Expectations](https://github.com/nodejs/admin/blob/main/MemberExpectations.md).
## Promoting content
We trust ambassadors to only request promotion of content that lines
up with the messages and topics outlined, that this content does not overly
market their employer or product/business, and that they have followed the review
process. An ambassador can request promotion of content in the following ways:
* Requesting that a post be reposted by the Node.js account on a social media platform.
* @-mentioning the nodejs/ambassadors team to ask other ambassadors to repost.
* Posting a link to the content in the "what's new" issue in nodejs/ambassadors
so that it goes out on the news feed.
For accounts managed by foundation staff, the staff will repost the social
media post without any need for validation based on the request coming from
an ambassador. For accounts managed by the project with an approval process,
(for example bluesky) documentation for the approval process will indicate
that repost requests from ambassadors should generally be approved. These
requests can be made through the existing social channel in the OpenJS Slack.
For that reason and for communication purposes and collaboration opportunities,
ambassadors should be members of the
[OpenJS Slack](https://slack-invite.openjsf.org/).
## Messages and topics to promote
### How to add messages or topics to promote
Messages or topics that ambassadors are asked to promote are added to this
document in the [Current messages for promotion](#current-messages-for-promotion)
section through the standard PR process except that they should be open
for 7 days before landing and should include and at mention to the
nodejs/TSC for awareness. They should be removed through the same process
when no longer relevant.
### Current messages for promotion
#### Sample message (Leave this one at the top)
##### Goal
The goal is to raise awareness of XYZ in the JavaScript ecosystem.
##### Related Links
List of links with more information about the topic to provide background
or the information to be shared.
##### Project contacts
Add a list of GitHub handles for those within the project that
have volunteered to be contacted when necessary by ambassadors
to get more info about the message to be promoted.
#### Node.js is a great choice for a JavaScript runtime
##### Goal
Highlight the benefits of choosing Node.js as your backend JavaScript runtime. Focus on what is great
about Node.js without drawing comparisons to alternatives. We don't want to say negative things about
other options, only highlight what is great about Node.js as a choice.
Some of the things to highlight include:
* How widely it is used (you never get fired for choosing Node.js).
* The openness of the project. It is part of the OpenJS Foundation and it's governance is set up to avoid
any one company from dominating the project. Decisions are made by the collaborators (of which there are quite
a few) versus a small number of people.
* It has predictable and stable releases and has delivered on the release schedule since 2015.
* It was a well defined security release process and manages security releases well.
* As the defacto standard, it has the highest likelihood of being supported for a given package on npm.
* It is not dependent on any one company for its continued existence reducing risk of using it.
* The large number of platforms supported.
* Asynchronous non-blocking i/o architecture drives high transactional throughput, making it ideal for web workloads.
* Single threaded programming model enables very low resource consumption, making it ideal for containerised workloads.
* Highly vibrant ecosystem with enterprise support from many vendors.
##### Related Links
* <https://github.com/nodejs/release>
* <https://github.com/nodejs/node/blob/main/doc/contributing/security-release-process.md>
* <https://github.com/nodejs/TSC/blob/main/TSC-Charter.md>
* <https://github.com/mhdawson/presentations/blob/main/2024/NodeConfEU_2024-Node.js_whats_next.pdf>
for slide usage and topping recent surveys.
##### Project contacts
* @mhdawson
#### How things get done in the Node.js project
##### Goal
Help people understand that no people are paid to answer their issues or
implement their pull requests. Things get done based on what volunteers work on
and the best way to get something fixed/changed is to submit a Pull request.
Some of the things to highlight include:
* Nobody is paid specifically to answer issues, fix bugs or implement new features.
* No company owns/supports Node.js. Most contributions are from individuals
as opposed to organizations. When an individual becomes a collaborator
the access and privileges are granted to the individual, their employer does
not gain any additional rights in the project.
* The governance of the project is specifically designed to prevent one or
a small number of companies from dominating the project.
* Decisions are made by the active collaborators, there is no single person
who makes a decision for the project on their own. This can slow down
decision making but most often results in better outcomes.
* The project is open and receptive to contributions. If you need something PR
in a fix or feature.
* Maintainers are people just like you, with many priorities and end goals. We
all have the same goal of moving the Node.js project forward but at the same
time we all have other responsibilities that affect how much time we have
available to do so.
* People are volunteering their time to review your PRs and answer questions in
the issues you open. Be mindfull of your asks for their time and acknowledge
the gift of their time. Too many issues/PRs in a short period of time may
overwhelm maintainers leading to less progress versus more, try to pace your
issues and PRs so that you don't have too many open at the same time. The
same goes for comments in discussions, try to avoid overwhelming a discussion
with too many responses, even too much useful data can overwhelm a discussion
leading to lower engagement.
* While volunteers work to do the right thing for the community, the project
does not owe anybody anything and does not tolerate abusive or
demanding language in issues, discussions or PRs. A respectful dialog will
maximize the chances of the outcome you desire.
* If you depend on timely support or an SLA, contract with a company that provides
paid support and will prioritize your issues.
##### Related Links
* <https://github.com/nodejs/TSC/blob/main/TSC-Charter.md#section-3-establishment-of-the-tsc>
##### Project contacts
* @mcollina
* @mhdawson
* @marco-ippolito
You can find their contact email in the [`README.md`](../../README.md#tsc-technical-steering-committee)
#### Node.js Type Stripping
##### Goal
The goal is to raise awareness of the Node.js TypeScript Type Stripping in the JavaScript ecosystem.
Some of the things to highlight include:
* The benefits and limitations of the current implementation.
* The `tsconfig.json` configuration options to use in combination for type checking.
* Updates on the implementation advancements.
##### Related Links
<!-- lint disable prohibited-strings remark-lint-->
* <https://github.com/nodejs/typescript>
* <https://nodejs.org/api/typescript.html>
* <https://nodejs.org/en/learn/typescript/run-natively>
* <https://satanacchio.hashnode.dev/everything-you-need-to-know-about-nodejs-type-stripping>
* <https://github.com/nodejs/amaro>
<!-- lint enable prohibited-strings remark-lint-->
##### Project contacts
* @marco-ippolito
#### Do I still need this dependency for my Node.js app?
##### Goal
Advancements over time in Node.js are improving the out of the box experience.
New versions are released all the time across Active LTS and Current development lines.
It's easy to miss something between the release notes and our busy work schedules.
Each of these on its own is respectable, but together they make a more cohesive narrative.
This also shows a [healthy ecosystem at work](https://brianmuenzenmeyer.com/posts/2024-do-i-need-this-node-dependency/#oss-pace-layers),
with projects learning from one another and their users.
"Recent" new or newish features, ordered by availability:
| Feature | Introduced | Release Status |
| ----------------------------------------------------------------------------------------------------- | ------------------------------------------------------ | ------------------------------------------------------------------ |
| [testing source code](https://nodejs.org/api/test.html) | [16.17.0](https://nodejs.org/en/blog/release/v16.17.0) | Stable as of 20.0.0 |
| [watching source code](https://nodejs.org/api/cli.html#--watch) | [16.19.0](https://nodejs.org/en/blog/release/v16.19.0) | Stable as of 20.13.0 |
| [parsing arguments](https://nodejs.org/api/util.html#utilparseargsconfig) | [18.3.0](https://nodejs.org/en/blog/release/v18.3.0) | Stable as of 20.0.0 |
| [reading environment](https://nodejs.org/api/cli.html#--env-fileconfig) | [20.6.0](https://nodejs.org/en/blog/release/v20.6.0) | Active Development |
| [styling output](https://nodejs.org/docs/latest-v22.x/api/util.html#utilstyletextformat-text-options) | [20.12.0](https://nodejs.org/en/blog/release/v20.12.0) | Stable, as of [22.13.0](https://github.com/nodejs/node/pull/56329) |
| [run scripts](https://nodejs.org/docs/latest/api/cli.html#--run) | [22.0.0](https://nodejs.org/en/blog/release/v22.0.0) | Stable, as of 22.0.0 |
| [run TypeScript](https://nodejs.org/api/cli.html#--experimental-strip-types) | [22.6.0](https://nodejs.org/en/blog/release/v22.6.0) | Active Development |
##### Related Links
<!-- lint disable prohibited-strings remark-lint-->
* <https://brianmuenzenmeyer.com/posts/2024-do-i-need-this-node-dependency/>
* <https://nodejs.org/api/test.html>
* <https://nodejs.org/api/cli.html#--watch>
* <https://nodejs.org/api/util.html#utilparseargsconfig>
* <https://nodejs.org/api/cli.html#--env-fileconfig>
* <https://nodejs.org/docs/latest-v22.x/api/util.html#utilstyletextformat-text-options>
* <https://nodejs.org/api/cli.html#--run>
* <https://nodejs.org/api/cli.html#--experimental-strip-types>
<!-- lint enable prohibited-strings remark-lint-->
##### Project contacts
* @bmuenzenmeyer

View File

@ -0,0 +1,305 @@
# Node.js API Documentation Tooling
The Node.js API documentation is generated by an in-house tooling that resides
within the [tools/doc](https://github.com/nodejs/node/tree/main/tools/doc)
directory.
The build process (using `make doc` or `make doc-only`) uses this tooling to
parse the markdown files in [`doc/api/`][] and generate the following:
1. Human-readable HTML in `out/doc/api/*.html`
2. A JSON representation in `out/doc/api/*.json`
These artifacts are published to nodejs.org for multiple versions of
Node.js. As an example the latest version of the human-readable HTML
is published to [nodejs.org/en/doc](https://nodejs.org/en/docs/),
and the latest version of the json documentation is published to
[nodejs.org/api/all.json](https://nodejs.org/api/all.json)
The artifacts are built as part of release builds by running the [doc-upload](https://github.com/nodejs/node/blob/1a83ad6a693f851199608ae957ac5d4f76871485/Makefile#L1218-L1224)
Makefile target as part of the release-sources part of the
iojs+release job.
This target runs the `doc` target to build the docs and then uses
`scp` to copy them onto the staging/www server into a directory of the form
`/home/staging/nodejs/<type>/<full_version>/docs` where <type> is e.g.
release, nightly, etc. The promotion step (either automatic for
nightlies or manual for releases) then moves the docs to
`/home/dist/nodejs/docs/\<full\_version>` where they are served by node.org.
**The key things to know about the tooling include:**
1. The entry-point is `tools/doc/generate.js`.
2. The tooling supports the CLI arguments listed in the table below.
3. The tooling processes one file at a time.
4. The tooling uses a set of dependencies as described in the dependencies
section.
5. The tooling parses the input files and does several transformations to the
AST (Abstract Syntax Tree).
6. The tooling generates a JSON output that contains the metadata and content of
the Markdown file.
7. The tooling generates a HTML output that contains a human-readable and ready
to-view version of the file.
This documentation serves the purpose of explaining the existing tooling
processes, to allow easier maintenance and evolution of the tooling. It is not
meant to be a guide on how to write documentation for Node.js.
#### Vocabulary & Good to Know's
* AST means "Abstract Syntax Tree" and it is a data structure that represents
the structure of a certain data format. In our case, the AST is a "graph"
representation of the contents of the Markdown file.
* MDN means [Mozilla Developer Network](https://developer.mozilla.org/en-US/)
and it is a website that contains documentation for web technologies. We use
it as a reference for the structure of the documentation.
* The
[Stability Index](https://nodejs.org/dist/latest/docs/api/documentation.html#stability-index)
is used to community the Stability of a given Node.js module. The Stability
levels include:
* Stability 0: Deprecated. (This module is Deprecated)
* Stability 1: Experimental. (This module is Experimental)
* Stability 2: Stable. (This module is Stable)
* Stability 3: Legacy. (This module is Legacy)
* Within Remark YAML snippets `<!-- something -->` are considered HTML nodes,
that's because YAML isn't valid Markdown content. (Doesn't abide by the
Markdown spec)
* "New Tooling" references to the (written from-scratch) API build tooling
introduced in `nodejs/nodejs.dev` that might replace the current one from
`nodejs/node`
## CLI Arguments
The tooling requires a `filename` argument and supports extra arguments (some
also required) as shown below:
| Argument | Description | Required | Example |
| --------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | -------- | ---------------------------------- |
| `--node-version=` | The version of Node.js that is being documented. It defaults to `process.version` which is supplied by Node.js itself | No | v19.0.0 |
| `--output-directory=` | The directory where the output files will be generated. | Yes | `./out/api/` |
| `--apilinks=` | This file is used as an index to specify the source file for each module | No | `./out/doc/api/apilinks.json` |
| `--versions-file=` | This file is used to specify an index of all previous versions of Node.js. It is used for the Version Navigation on the API docs page. | No | `./out/previous-doc-versions.json` |
**Note:** both of the `apilinks` and `versions-file` parameters are generated by
the Node.js build process (Makefile). And they're files containing a JSON
object.
### Basic Usage
```bash
# cd tools/doc
npm run node-doc-generator ${filename}
```
**OR**
```bash
# nodejs/node root directory
make doc
```
## Dependencies and how the Tooling works internally
The API tooling uses an-AST-alike library called
[unified](https://github.com/unifiedjs/unified) for processing the Input file as
a Graph that supports easy modification and update of its nodes.
In addition to `unified` we also use
[Remark](https://github.com/remarkjs/remark) for manipulating the Markdown part,
and [Rehype](https://github.com/rehypejs/rehype) to help convert to and from
Markdown.
### What are the steps of the internal tooling?
The tooling uses `unified` pipe-alike engine to pipe each part of the process.
(The description below is a simplified version)
* Starting from reading the Frontmatter section of the Markdown file with
[remark-frontmatter](https://www.npmjs.com/package/remark-frontmatter).
* Then the tooling goes to parse the Markdown by using `remark-parse` and adds
support to [GitHub Flavoured Markdown](https://github.github.com/gfm/).
* The tooling proceeds by parsing some of the Markdown nodes and transforming
them to HTML.
* The tooling proceeds to generate the JSON output of the file.
* Finally it does its final node transformations and generates a stringified
HTML.
* It then stores the output to a JSON file and adds extra styling to the HTML
and then stores the HTML file.
### What each file is responsible for?
The files listed below are the ones referenced and actually used during the
build process of the API docs as we see on <https://nodejs.org/api>. The
remaining files from the directory might be used by other steps of the Node.js
Makefile or might even be deprecated/remnant of old processes and might need to
be revisited/removed.
* **`html.mjs`**: Responsible for transforming nodes by decorating them with
visual artifacts for the HTML pages;
* For example, transforming man or JS doc references to links correctly
referring to respective External documentation.
* **`json.mjs`**: Responsible for generating the JSON output of the file;
* It is mostly responsible for going through the whole Markdown file and
generating a JSON object that represent the Metadata of a specific Module.
* For example, for the FS module, it will generate an object with all its
methods, events, classes and use several regular expressions (ReGeX) for
extracting the information needed.
* **`generate.mjs`**: Main entry-point of doc generation for a specific file. It
does e2e processing of a documentation file;
* **`allhtml.mjs`**: A script executed after all files are generated to create a
single "all" page containing all the HTML documentation;
* **`alljson.mjs`**: A script executed after all files are generated to create a
single "all" page containing all the JSON entries;
* **`markdown.mjs`**: Contains utility to replace Markdown links to work with
the <https://nodejs.org/api/> website.
* **`common.mjs`**: Contains a few utility functions that are used by the other
files.
* **`type-parser.mjs`**: Used to replace "type references" (e.g. "String", or
"Buffer") to the correct Internal/External documentation pages (i.e. MDN or
other Node.js documentation pages).
**Note:** It is important to mention that other files not mentioned here might
be used during the process but are not relevant to the generation of the API
docs themselves. You will notice that a lot of the logic within the build
process is **specific** to the current <https://nodejs.org/api/> infrastructure.
Just as adding some JavaScript snippets, styles, transforming certain Markdown
elements into HTML, and adding certain HTML classes or such things.
**Note:** Regarding the previous **Note** it is important to mention that we're
currently working on an API tooling that is generic and independent of the
current Nodejs.org Infrastructure.
[The new tooling that is functional is available at the nodejs.dev repository](https://github.com/nodejs/nodejs.dev/blob/main/scripts/syncApiDocs.js)
and uses plain ReGeX (No AST) and [MDX](https://mdxjs.com/).
## The Build Process
The build process that happens on `generate.mjs` follows the steps below:
* Links within the Markdown are replaced directly within the source Markdown
(AST) (`markdown.replaceLinks`)
* This happens within `markdown.mjs` and basically it adds suffixes or
modifies link references within the Markdown
* This is necessary for the `https://nodejs.org` infrastructure as all pages
are suffixed with `.html`
* Text (and some YAML) Nodes are transformed/modified through
`html.preprocessText`
* JSON output is generated through `json.jsonAPI`
* The title of the page is inferred through `html.firstHeader`
* Nodes are transformed into HTML Elements through `html.preprocessElements`
* The HTML Table of Contents (ToC) is generated through `html.buildToc`
### `html.mjs`
This file is responsible for doing node AST transformations that either update
Markdown nodes to decorate them with more data or transform them into HTML Nodes
that attain a certain visual responsibility; For example, to generate the "Added
at" label, or the Source Links or the Stability Index, or the History table.
**Note:** Methods not listed below are either not relevant or utility methods
for string/array/object manipulation (e.g.: are used by the other methods
mentioned below).
#### `preprocessText`
**New Tooling:** Most of the features within this method are available within
the new tooling.
This method does two things:
* Replaces the Source Link YAML entry `<-- source_link= -->` into a "Source
Link" HTML anchor element.
* Replaces type references within the Markdown (text) (i.e.: "String", "Buffer")
into the correct HTML anchor element that links to the correct documentation
page.
* The original node then gets mutated from text to HTML.
* It also updates references to Linux "MAN" pages to Web versions of them.
#### `firstHeader`
**New Tooling:** All features within this method are available within the new
Tooling.
Is used to attempt to extract the first heading of the page (recursively) to
define the "title" of the page.
**Note:** As all API Markdown files start with a Heading, this could possibly be
improved to a reduced complexity.
#### `preprocessElements`
**New Tooling:** All features within this method are available within the new
tooling.
This method is responsible for doing multiple transformations within the AST
Nodes, in majority, transforming the source node in respective HTML elements
with diverse responsibilities, such as:
* Updating Markdown `code` blocks by adding Language highlighting
* It also adds the "CJS"/"MJS" switch to Nodes that are followed by their
CJS/ESM equivalents.
* Increasing the Heading level of each Heading
* Parses YAML blocks and transforms them into HTML elements (See more at the
`parseYAML` method)
* Updates BlockQuotes that are prefixed by the "Stability" word into a Stability
Index HTML element.
#### `parseYAML`
**New Tooling:** Most of the features within this method are available within
the new tooling.
This method is responsible for parsing the `<--YAML snippets -->` and
transforming them into HTML elements.
It follows a certain kind of "schema" that basically constitutes in the
following options:
| YAML Key | Description | Example | Example Result | Available on new tooling |
| ------------- | ------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------- | --------------------------- | ------------------------ |
| `added` | It's used to reference when a certain "module", "class" or "method" was added on Node.js | `added: v0.1.90` | `Added in: v0.1.90` | Yes |
| `deprecated` | It's used to reference when a certain "module", "class" or "method" was deprecated on Node.js | `deprecated: v0.1.90` | `Deprecated since: v0.1.90` | Yes |
| `removed` | It's used to reference when a certain "module", "class" or "method" was removed on Node.js | `removed: v0.1.90` | `Removed in: v0.1.90` | No |
| `changes` | It's used to describe all the changes (historical ones) that happened within a certain "module", "class" or "method" in Node.js | `[{ version: v0.1.90, pr-url: '', description: '' }]` | -- | Yes |
| `napiVersion` | It's used to describe in which version of the N-API this "module", "class" or "method" is available within Node.js | `napiVersion: 1` | `N-API version: 1` | Yes |
**Note:** The `changes` field gets prepended with the `added`, `deprecated` and
`removed` fields if they exist. The table only gets generated if a `changes`
field exists. In the new tooling only "added" is prepended for now.
#### `buildToc`
**New Tooling:** This feature is natively available within the new tooling
through MDX.
This method generates the Table of Contents based on all the Headings of the
Markdown file.
#### `altDocs`
**New Tooling:** All features within this method are available within the new
tooling.
This method generates a version picker for the current page to be shown in older
versions of the API docs.
### `json.mjs`
This file is responsible for generating a JSON object that (supposedly) is used
for IDE-Intellisense or for indexing of all the "methods", "classes", "modules",
"events", "constants" and "globals" available within a certain Markdown file.
It attempts a best effort extraction of the data by using several regular
expression patterns (ReGeX).
**Note:** JSON output generation is currently not supported by the new tooling,
but it is in the pipeline for development.
#### `jsonAPI`
This method traverses all the AST Nodes by iterating through each one of them
and infers the kind of information each node contains through ReGeX. Then it
mutate the data and appends it to the final JSON object.
For a more in-depth information we recommend to refer to the `json.mjs` file as
it contains a lot of comments.
[`doc/api/`]: https://github.com/nodejs/node/tree/main/doc/api

View File

@ -0,0 +1,104 @@
# How to backport a pull request to a release line
## Staging branches
Each release line has a staging branch that serves as a workspace for preparing releases.
The branch format is `vN.x-staging`, where `N` is the major release number.
For active staging branches, refer to the [Release Schedule][].
## Identifying changes that require a backport
If a cherry-pick from `main` doesn't apply cleanly on a staging branch, the pull request
will be labeled for the release line (e.g., `backport-requested-vN.x`). This indicates
that manual backporting is required.
## Criteria for backporting
The "Current" release line is more flexible than LTS lines. LTS branches, detailed in the [Release Plan][],
require commits to mature in the Current release for at least two weeks before backporting.
## Labeling backport issues and PRs
Use the following labels, with `N` in `vN.x` denoting the major release number:
| Label | Description |
| ----------------------- | ------------------------------------------------------------------- |
| backport-blocked-vN.x | PRs for `vN.x-staging` blocked by pending backports from other PRs. |
| backport-open-vN.x | Indicates an open backport for the PR. |
| backport-requested-vN.x | PR awaiting manual backport to `vN.x-staging`. |
| backported-to-vN.x | PR successfully backported to `vN.x-staging`. |
| baking-for-lts | PRs awaiting LTS release after maturation in Current. |
| lts-watch-vN.x | PRs possibly included in `vN.x` LTS releases. |
| vN.x | Issues or PRs impacting `vN.x-staging` (or reproducible on vN.x). |
## Submitting a backport pull request
For the following steps, let's assume that you need to backport PR `123`
to the vN.x release line. All commands will use the `vN.x-staging` branch
as the target branch. In order to submit a backport pull request to another
branch, simply replace `N` with the version number for the targeted release
line.
### Automated process
1. Ensure [`@node-core/utils`][] is installed.
2. Execute [`git node backport`][] command:
```bash
# Example: Backport PR 123 to vN.x-staging
git node backport 123 --to=N
```
3. Proceed to step 5 in the Manual section below.
### Manual process
1. Checkout the `vN.x-staging` branch.
2. Verify that the local staging branch is up to date with the remote.
3. Create a new branch based on `vN.x-staging`:
```bash
git fetch upstream vN.x-staging:vN.x-staging -f
git checkout -b backport-123-to-vN.x vN.x-staging
```
4. Cherry-pick the desired commit(s):
```bash
git cherry-pick <commit hash>
```
5. Resolve conflicts using `git add` and `git cherry-pick --continue`.
6. Leave the commit message as is. If you think it should be modified, comment
in the pull request. The `Backport-PR-URL` metadata does need to be added to
the commit, but this will be done later.
7. Verify that `make -j4 test` passes.
8. Push the changes to your fork.
9. Open a pull request:
* Target `vN.x-staging`.
* Title format: `[vN.x backport] <commit title>` (e.g., `[v20.x backport] process: improve performance of nextTick`).
* Reference the original PR in the description.
10. Update the `backport-requested-vN.x` label on the original pull request to `backport-open-vN.x`.
11. If conflicts arise during the review process, the following command be used to rebase:
```bash
git pull --rebase upstream vN.x-staging
```
Once merged, update the original PR's label from `backport-open-vN.x` to `backported-to-vN.x`.
[Release Plan]: https://github.com/nodejs/Release#release-plan
[Release Schedule]: https://github.com/nodejs/Release#release-schedule
[`@node-core/utils`]: https://github.com/nodejs/node-core-utils
[`git node backport`]: https://github.com/nodejs/node-core-utils/blob/main/docs/git-node.md#git-node-backport

View File

@ -0,0 +1,62 @@
# Building Node.js with Ninja
The purpose of this guide is to show how to build Node.js using [Ninja][], as
doing so can be significantly quicker than using `make`. Please see
[Ninja's site][Ninja] for installation instructions (Unix only).
[Ninja][] is supported in the Makefile. Run `./configure --ninja` to configure
the project to run the regular `make` commands with Ninja.
When modifying only the JS layer in `lib`, you can use:
```bash
./configure --ninja --node-builtin-modules-path "$(pwd)"
```
For example, `make` will execute `ninja -C out/Release` internally
to produce a compiled release binary, It will also execute
`ln -fs out/Release/node node`, so that you can execute `./node` at
the project's root.
When running `make`, you will see output similar to the following
if the build has succeeded:
```console
ninja: Entering directory `out/Release`
[4/4] LINK node, POSTBUILDS
```
The bottom line will change while building, showing the progress as
`[finished/total]` build steps. This is useful output that `make` does not
produce and is one of the benefits of using Ninja. When using Ninja, builds
are always run in parallel, based by default on the number of CPUs your
system has. You can use the `-j` parameter to override this behavior,
which is equivalent to the `-j` parameter in the regular `make`:
```bash
make -j4 # With this flag, Ninja will limit itself to 4 parallel jobs,
# regardless of the number of cores on the current machine.
```
Note: if you are on macOS and use GNU Make version `3.x`, the `-jn` flag
will not work. You can either upgrade to `v4.x` (e.g. using a package manager
such as [Homebrew](https://formulae.brew.sh/formula/make#default)) or use `make JOBS=n`.
## Producing a debug build
To create a debug build rather than a release build:
```bash
./configure --ninja --debug && make
```
## Customizing `ninja` path
On some systems (such as RHEL7 and below), the Ninja binary might be installed
with a different name. For these systems use the `NINJA` env var:
```bash
./configure --ninja && NINJA="ninja-build" make
```
[Ninja]: https://ninja-build.org/

View File

@ -0,0 +1,43 @@
# Code of Conduct
The Node.js project has a [Code of Conduct][] that _all_ contributors are
expected to follow. This code describes the _minimum_ behavior expectations
for all contributors.
As a contributor to Node.js, how you choose to act and interact towards your
fellow contributors, as well as to the community, will reflect back not only
on yourself but on the project as a whole. The Code of Conduct is designed and
intended, above all else, to help establish a culture within the project that
allows anyone and everyone who wants to contribute to feel safe doing so.
Should any individual act in any way that is considered in violation of the
[Code of Conduct][], corrective actions will be taken. It is possible, however,
for any individual to _act_ in such a manner that is not in violation of the
strict letter of the Code of Conduct guidelines while still going completely
against the spirit of what that Code is intended to accomplish.
Open, diverse, and inclusive communities live and die on the basis of trust.
Contributors can disagree with one another so long as they trust that those
disagreements are in good faith and everyone is working towards a common goal.
## Bad actors
All contributors to Node.js tacitly agree to abide by both the letter and
spirit of the [Code of Conduct][]. Failure, or unwillingness, to do so will
result in contributions being respectfully declined.
A _bad actor_ is someone who repeatedly violates the spirit of the Code of
Conduct through failure to regulate how they
interact with others. In doing so, bad actors
alienate other contributors, discourage collaboration, and generally reflect
poorly on the project as a whole.
Being a bad actor may be intentional or unintentional. Typically, unintentional
bad behavior can be easily corrected by being quick to apologize and correct
course _even if you are not entirely convinced you need to_. Giving other
contributors the benefit of the doubt and having a sincere willingness to admit
that you _might_ be wrong is critical for any successful open collaboration.
Don't be a bad actor.
[Code of Conduct]: https://github.com/nodejs/admin/blob/HEAD/CODE_OF_CONDUCT.md

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,122 @@
# Commit queue
_tl;dr: You can land pull requests by adding the `commit-queue` label to it._
Commit Queue is a feature for the project which simplifies the
landing process by automating it via GitHub Actions. With it, collaborators can
land pull requests by adding the `commit-queue` label to a PR. All
checks will run via `@node-core/utils`, and if the pull request is ready to
land, the Action will rebase it and push to `main`.
This document gives an overview of how the Commit Queue works, as well as
implementation details, reasoning for design choices, and current limitations.
## Overview
From a high-level, the Commit Queue works as follow:
1. Collaborators will add `commit-queue` label to pull requests ready to land
2. Every five minutes the queue will do the following for each mergeable pull request
with the label:
1. Check if the PR also has a `request-ci` label (if it has, skip this PR
since it's pending a CI run)
2. Check if the last Jenkins CI is finished running (if it is not, skip this
PR)
3. Remove the `commit-queue` label
4. Run `git node land <pr> --oneCommitMax`
5. If it fails:
1. Abort `git node land` session
2. Add `commit-queue-failed` label to the PR
3. Leave a comment on the PR with the output from `git node land`
4. Skip next steps, go to next PR in the queue
6. If it succeeds:
1. Push the changes to nodejs/node
2. Leave a comment on the PR with `Landed in ...`
3. Close the PR
4. Go to next PR in the queue
To make the Commit Queue squash all the commits of a pull request into the
first one, add the `commit-queue-squash` label.
To make the Commit Queue land a pull request containing several commits, add the
`commit-queue-rebase` label. When using this option, make sure
that all commits are self-contained, meaning every commit should pass all tests.
## Current limitations
The Commit Queue feature is still in early stages, and as such it might not
work for more complex pull requests. These are the currently known limitations
of the commit queue:
1. All commits in a pull request must either be following commit message
guidelines or be a valid [`fixup!`](https://git-scm.com/docs/git-commit#Documentation/git-commit.txt---fixupamendrewordltcommitgt)
commit that will be correctly handled by the [`--autosquash`](https://git-scm.com/docs/git-rebase#Documentation/git-rebase.txt---autosquash)
option
2. A CI must've ran and succeeded since the last change on the PR
3. A collaborator must have approved the PR since the last change
4. Only Jenkins CI and GitHub Actions are checked (V8 CI and CITGM are ignored)
## Implementation
The [action](../../.github/workflows/commit-queue.yml) will run on scheduler
events every five minutes. Five minutes is the smallest number accepted by
the scheduler. The scheduler is not guaranteed to run every five minutes, it
might take longer between runs.
Using the scheduler is preferable over using pull\_request\_target for two
reasons:
1. if two Commit Queue Actions execution overlap, there's a high-risk that
the last one to finish will fail because the local branch will be out of
sync with the remote after the first Action pushes. `issue_comment` event
has the same limitation.
2. `pull_request_target` will only run if the Action exists on the base commit
of a pull request, and it will run the Action version present on that
commit, meaning we wouldn't be able to use it for already opened PRs
without rebasing them first.
`@node-core/utils` is configured with a personal token and
a Jenkins token from
[@nodejs-github-bot](https://github.com/nodejs/github-bot).
`octokit/graphql-action` is used to fetch all pull requests with the
`commit-queue` label. The output is a JSON payload, so `jq` is used to turn
that into a list of PR ids we can pass as arguments to
[`commit-queue.sh`](../../tools/actions/commit-queue.sh).
> The personal token only needs permission for public repositories and to read
> profiles, we can use the GITHUB\_TOKEN for write operations. Jenkins token is
> required to check CI status.
`commit-queue.sh` receives the following positional arguments:
1. The repository owner
2. The repository name
3. The Action GITHUB\_TOKEN
4. Every positional argument starting at this one will be a pull request ID of
a pull request with commit-queue set.
The script will iterate over the pull requests. `ncu-ci` is used to check if
the last CI is still pending, and calls to the GitHub API are used to check if
the PR is waiting for CI to start (`request-ci` label). The PR is skipped if CI
is pending. No other CI validation is done here since `git node land` will fail
if the last CI failed.
The script removes the `commit-queue` label. It then runs `git node land`,
forwarding stdout and stderr to a file. If any errors happen,
`git node land --abort` is run, and then a `commit-queue-failed` label is added
to the PR, as well as a comment with the output of `git node land`.
If no errors happen during `git node land`, the script will use the
`GITHUB_TOKEN` to push the changes to `main`, and then will leave a
`Landed in ...` comment in the PR, and then will close it. Iteration continues
until all PRs have done the steps above.
## Reverting broken commits
Reverting broken commits is done manually by collaborators, just like when
commits are landed manually via `git node land`. An easy way to revert is a
good feature for the project, but is not explicitly required for the Commit
Queue to work because the Action lands PRs just like collaborators do today. If
once we start using the Commit Queue we notice that the number of required
reverts increases drastically, we can pause the queue until a Revert Queue is
implemented, but until then we can enable the Commit Queue and then work on a
Revert Queue as a follow-up.

View File

@ -0,0 +1,58 @@
# To be or not to be in core
This document explains things to consider when deciding whether a component
should be in core or not.
A component may be included in core as a dependency, a module, or integrated
into the code base. The same arguments for including/not including in core
generally apply in all of these cases.
## Strong arguments for including a component in core
1. The component provides functionality that is standardized (such as a
[Web API][]) and overlaps with existing functionality.
2. The component can only be implemented in core.
3. The component can only be implemented in a performant way in core.
4. Developer experience is significantly improved if the component is in core.
5. The component provides functionality that can be expected to solve at
least one common use case Node.js users face.
6. The component requires native bindings. Inclusion in core enables
utility across operating systems and architectures without requiring
users to have a native compilation toolchain.
7. Part or all of the component will also be re-used or duplicated in core.
## Strong arguments against including a component in core
1. None of the arguments listed in the previous section apply.
2. The component has a license that prohibits Node.js from including it in core
without also changing its own license.
3. There is already similar functionality in core and adding the component will
provide a second API to do the same thing.
4. A component (or/and the standard it is based on) is deprecated and there is
a non-deprecated alternative.
5. The component is evolving quickly and inclusion in core will require frequent
API changes.
## Benefits and challenges
When it is unclear whether a component should be included in core, it might be
helpful to consider these additional factors.
### Benefits
1. The component will receive more frequent testing with Node.js CI and CITGM.
2. The component will be integrated into the LTS workflow.
3. Documentation will be integrated with core.
4. There is no dependency on npm.
### Challenges
1. Inclusion in core, rather than as an ecosystem module, is likely to reduce
code merging velocity. The Node.js process for code review and merging is
more time-consuming than that of most separate modules.
2. By being bound to the Node.js release cycle, it is harder and slower to
publish patches.
3. Less flexibility for users. They can't update the component
when they choose without also updating Node.js.
[Web API]: https://developer.mozilla.org/en-US/docs/Web/API

View File

@ -0,0 +1,407 @@
# C++ style guide
See also the [C++ codebase README](../../src/README.md) for C++ idioms in the
Node.js codebase not related to stylistic issues.
## Table of contents
* [Guides and references](#guides-and-references)
* [Formatting](#formatting)
* [Left-leaning (C++ style) asterisks for pointer declarations](#left-leaning-c-style-asterisks-for-pointer-declarations)
* [C++ style comments](#c-style-comments)
* [2 spaces of indentation for blocks or bodies of conditionals](#2-spaces-of-indentation-for-blocks-or-bodies-of-conditionals)
* [4 spaces of indentation for statement continuations](#4-spaces-of-indentation-for-statement-continuations)
* [Align function arguments vertically](#align-function-arguments-vertically)
* [Initialization lists](#initialization-lists)
* [PascalCase for methods, functions, and classes](#pascalcase-for-methods-functions-and-classes)
* [`snake_case` for local variables and parameters](#snake_case-for-local-variables-and-parameters)
* [`snake_case_` for private class fields](#snake_case_-for-private-class-fields)
* [`snake_case` for C-like structs](#snake_case-for-c-like-structs)
* [Space after `template`](#space-after-template)
* [Memory management](#memory-management)
* [Memory allocation](#memory-allocation)
* [Use `nullptr` instead of `NULL` or `0`](#use-nullptr-instead-of-null-or-0)
* [Use explicit pointer comparisons](#use-explicit-pointer-comparisons)
* [Ownership and smart pointers](#ownership-and-smart-pointers)
* [Avoid non-const references](#avoid-non-const-references)
* [Use AliasedBuffers to manipulate TypedArrays](#use-aliasedbuffers-to-manipulate-typedarrays)
* [Others](#others)
* [Type casting](#type-casting)
* [Using `auto`](#using-auto)
* [Do not include `*.h` if `*-inl.h` has already been included](#do-not-include-h-if--inlh-has-already-been-included)
* [Avoid throwing JavaScript errors in C++ methods](#avoid-throwing-javascript-errors-in-c)
* [Avoid throwing JavaScript errors in nested C++ methods](#avoid-throwing-javascript-errors-in-nested-c-methods)
## Guides and references
The Node.js C++ codebase strives to be consistent in its use of language
features and idioms, as well as have some specific guidelines for the use of
runtime features.
Coding guidelines are based on the following guides (highest priority first):
1. This document.
2. The [Google C++ Style Guide][].
3. The ISO [C++ Core Guidelines][].
In general, code should follow the C++ Core Guidelines, unless overridden by the
Google C++ Style Guide or this document. At the moment these guidelines are
checked manually by reviewers with the goal to validate this with automatic
tools.
## Formatting
Unfortunately, the C++ linter (based on [Google's `cpplint`][]), which can be
run explicitly via `make lint-cpp`, does not currently catch a lot of rules that
are specific to the Node.js C++ code base. This document explains the most
common of these rules:
### Left-leaning (C++ style) asterisks for pointer declarations
`char* buffer;` instead of `char *buffer;`
### C++ style comments
Use C++ style comments (`//`) for both single-line and multi-line comments.
Comments should also start with uppercase and finish with a dot.
Examples:
```cpp
// A single-line comment.
// Multi-line comments
// should also use C++
// style comments.
```
The codebase may contain old C style comments (`/* */`) from before this was the
preferred style. Feel free to update old comments to the preferred style when
working on code in the immediate vicinity or when changing/improving those
comments.
### 2 spaces of indentation for blocks or bodies of conditionals
```cpp
if (foo)
bar();
```
or
```cpp
if (foo) {
bar();
baz();
}
```
Braces are optional if the statement body only has one line.
`namespace`s receive no indentation on their own.
### 4 spaces of indentation for statement continuations
```cpp
VeryLongTypeName very_long_result = SomeValueWithAVeryLongName +
SomeOtherValueWithAVeryLongName;
```
Operators are before the line break in these cases.
### Align function arguments vertically
```cpp
void FunctionWithAVeryLongName(int parameter_with_a_very_long_name,
double other_parameter_with_a_very_long_name,
...);
```
If that doesn't work, break after the `(` and use 4 spaces of indentation:
```cpp
void FunctionWithAReallyReallyReallyLongNameSeriouslyStopIt(
int okay_there_is_no_space_left_in_the_previous_line,
...);
```
### Initialization lists
Long initialization lists are formatted like this:
```cpp
HandleWrap::HandleWrap(Environment* env,
Local<Object> object,
uv_handle_t* handle,
AsyncWrap::ProviderType provider)
: AsyncWrap(env, object, provider),
state_(kInitialized),
handle_(handle) {
```
### PascalCase for methods, functions, and classes
Exceptions are simple getters/setters, which are named `property_name()` and
`set_property_name()`, respectively.
```cpp
class FooBar {
public:
void DoSomething();
static void DoSomethingButItsStaticInstead();
void set_foo_flag(int flag_value);
int foo_flag() const; // Use const-correctness whenever possible.
};
```
### `snake_case` for local variables and parameters
```cpp
int FunctionThatDoesSomething(const char* important_string) {
const char* pointer_into_string = important_string;
}
```
### `snake_case_` for private class fields
```cpp
class Foo {
private:
int counter_ = 0;
};
```
### `snake_case` for C-like structs
For plain C-like structs snake\_case can be used.
```cpp
struct foo_bar {
int name;
};
```
### Space after `template`
```cpp
template <typename T>
class FancyContainer {
...
};
```
## Memory management
### Memory allocation
* `Malloc()`, `Calloc()`, etc. from `util.h` abort in Out-of-Memory situations
* `UncheckedMalloc()`, etc. return `nullptr` in OOM situations
### Use `nullptr` instead of `NULL` or `0`
Further reading in the [C++ Core Guidelines][ES.47].
### Use explicit pointer comparisons
Use explicit comparisons to `nullptr` when testing pointers, i.e.
`if (foo == nullptr)` instead of `if (foo)` and
`foo != nullptr` instead of `!foo`.
### Ownership and smart pointers
* [R.20][]: Use `std::unique_ptr` or `std::shared_ptr` to represent ownership
* [R.21][]: Prefer `unique_ptr` over `shared_ptr` unless you need to share
ownership
Use `std::unique_ptr` to make ownership transfer explicit. For example:
```cpp
std::unique_ptr<Foo> FooFactory();
void FooConsumer(std::unique_ptr<Foo> ptr);
```
Since `std::unique_ptr` has only move semantics, passing one by value transfers
ownership to the callee and invalidates the caller's instance.
Don't use `std::auto_ptr`, it is deprecated ([Reference][cppref_auto_ptr]).
### Avoid non-const references
Using non-const references often obscures which values are changed by an
assignment. Consider using a pointer instead, which requires more explicit
syntax to indicate that modifications take place.
```cpp
class ExampleClass {
public:
explicit ExampleClass(OtherClass* other_ptr) : pointer_to_other_(other_ptr) {}
void SomeMethod(const std::string& input_param,
std::string* in_out_param); // Pointer instead of reference
const std::string& get_foo() const { return foo_string_; }
void set_foo(const std::string& new_value) { foo_string_ = new_value; }
void ReplaceCharacterInFoo(char from, char to) {
// A non-const reference is okay here, because the method name already tells
// users that this modifies 'foo_string_' -- if that is not the case,
// it can still be better to use an indexed for loop, or leave appropriate
// comments.
for (char& character : foo_string_) {
if (character == from)
character = to;
}
}
private:
std::string foo_string_;
// Pointer instead of reference. If this object 'owns' the other object,
// this should be a `std::unique_ptr<OtherClass>`; a
// `std::shared_ptr<OtherClass>` can also be a better choice.
OtherClass* pointer_to_other_;
};
```
### Use AliasedBuffers to manipulate TypedArrays
When working with typed arrays that involve direct data modification
from C++, use an `AliasedBuffer` when possible. The API abstraction and
the usage scope of `AliasedBuffer` are documented in
[aliased\_buffer.h][aliased_buffer.h].
```cpp
// Create an AliasedBuffer.
AliasedBuffer<uint32_t, v8::Uint32Array> data;
...
// Modify the data through natural operator semantics.
data[0] = 12345;
```
## Others
### Type casting
* Use `static_cast<T>` if casting is required, and it is valid.
* Use `reinterpret_cast` only when it is necessary.
* Avoid C-style casts (`(type)value`).
* `dynamic_cast` does not work because Node.js is built without
[Run Time Type Information][].
Further reading:
* [ES.48][]: Avoid casts
* [ES.49][]: If you must use a cast, use a named cast
### Using `auto`
Being explicit about types is usually preferred over using `auto`.
Use `auto` to avoid type names that are noisy, obvious, or unimportant. When
doing so, keep in mind that explicit types often help with readability and
verifying the correctness of code.
```cpp
for (const auto& item : some_map) {
const KeyType& key = item.first;
const ValType& value = item.second;
// The rest of the loop can now just refer to key and value,
// a reader can see the types in question, and we've avoided
// the too-common case of extra copies in this iteration.
}
```
### Do not include `*.h` if `*-inl.h` has already been included
Do:
```cpp
#include "util-inl.h" // already includes util.h
```
Instead of:
```cpp
#include "util.h"
#include "util-inl.h"
```
### Avoid throwing JavaScript errors in C++
When there is a need to throw errors from a C++ binding method, try to
return the data necessary for constructing the errors to JavaScript,
then construct and throw the errors [using `lib/internal/errors.js`][errors].
In general, type-checks on arguments should be done in JavaScript
before the arguments are passed into C++. Then in the C++ binding, simply using
`CHECK` assertions to guard against invalid arguments should be enough.
If the return value of the binding cannot be used to signal failures or return
the necessary data for constructing errors in JavaScript, pass a context object
to the binding and put the necessary data inside in C++. For example:
```cpp
void Foo(const FunctionCallbackInfo<Value>& args) {
Environment* env = Environment::GetCurrent(args);
// Let the JavaScript handle the actual type-checking,
// only assertions are placed in C++
CHECK_EQ(args.Length(), 2);
CHECK(args[0]->IsString());
CHECK(args[1]->IsObject());
int err = DoSomethingWith(args[0].As<String>());
if (err) {
// Put the data inside the error context
Local<Object> ctx = args[1].As<Object>();
Local<String> key = FIXED_ONE_BYTE_STRING(env->isolate(), "code");
ctx->Set(env->context(), key, err).FromJust();
} else {
args.GetReturnValue().Set(something_to_return);
}
}
// In the initialize function
env->SetMethod(target, "foo", Foo);
```
```js
exports.foo = function(str) {
// Prefer doing the type-checks in JavaScript
if (typeof str !== 'string') {
throw new errors.codes.ERR_INVALID_ARG_TYPE('str', 'string');
}
const ctx = {};
const result = binding.foo(str, ctx);
if (ctx.code !== undefined) {
throw new errors.codes.ERR_ERROR_NAME(ctx.code);
}
return result;
};
```
#### Avoid throwing JavaScript errors in nested C++ methods
When you need to throw a JavaScript exception from C++ (i.e.
`isolate()->ThrowException()`), do it as close to the return to JavaScript as
possible, and not inside of nested C++ calls. Since this changes the JavaScript
execution state, doing it closest to where it is consumed reduces the chances of
side effects.
Node.js is built [without C++ exception handling][], so code using `throw` or
even `try` and `catch` **will** break.
[C++ Core Guidelines]: https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines
[ES.47]: https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Res-nullptr
[ES.48]: https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Res-casts
[ES.49]: https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Res-casts-named
[Google C++ Style Guide]: https://google.github.io/styleguide/cppguide.html
[Google's `cpplint`]: https://github.com/google/styleguide
[R.20]: https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Rr-owner
[R.21]: https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Rr-unique
[Run Time Type Information]: https://en.wikipedia.org/wiki/Run-time_type_information
[aliased_buffer.h]: https://github.com/nodejs/node/blob/HEAD/src/aliased_buffer.h#L12
[cppref_auto_ptr]: https://en.cppreference.com/w/cpp/memory/auto_ptr
[errors]: https://github.com/nodejs/node/blob/HEAD/doc/contributing/using-internal-errors.md
[without C++ exception handling]: https://gcc.gnu.org/onlinedocs/libstdc++/manual/using_exceptions.html#intro.using.exception.no

View File

@ -0,0 +1,144 @@
# Diagnostic tooling support tiers
Diagnostic tooling is important to the consumers of Node.js. It is used both
in development and in production in order to investigate problems. The failure
of one of these tools may be as big a problem for an end user as a bug within
the runtime itself.
The Node.js project has assessed the tools and the APIs which support those
tools. Each of the tools and APIs has been put into one of
the following tiers.
* Tier 1 - Must always be working (CI tests passing) for all
Current and LTS Node.js releases. A release will not be shipped if the test
suite for the tool/API is not green. To be considered for inclusion
in this tier it must have a good test suite and that test suite and a job
must exist in the Node.js CI so that it can be run as part of the release
process. Tests on `main` will be run nightly when possible to provide
early warning of potential issues. No commit to the current and LTS
release branches should break this tool/API if the next major release
is within 1 month. In addition:
* The maintainers of the tool must remain responsive when there
are problems;
* The tool must be actively used by the ecosystem;
* The tool must be heavily depended on;
* The tool must have a guide or other documentation in the Node.js GitHub
organization or website;
* The tool must be working on all supported platforms;
* The tool must only be using APIs exposed by Node.js as opposed to
its dependencies;
* The tool must be open source.
* Tier 2 - Must be working (CI tests passing) for all
LTS releases. An LTS release will not be shipped if the test
suite for the tool/API is not green. To be considered for inclusion
in this tier it must have a good test suite and that test suite and a job
must exist in the Node.js CI so that it can be run as part of the release
process. In addition:
* The maintainers of the tool must remain responsive when
there are problems;
* The tool must be actively used by the ecosystem;
* The tool must be heavily depended on;
* The tool must have a guide or other documentation in the Node.js GitHub
organization or website;
* The tool must be open source.
* Tier 3 - If possible its test suite will be run at least nightly
in the Node.js CI or in CITGM, and issues opened for failures.
Does not block shipping a release.
* Tier 4 - Does not block shipping a release.
* Unclassified - tool/API is new or does not have the required testing in the
Node.js CI in order to qualify for a higher tier.
The choice of which tier a particular tool will be assigned to, will be a
collaborative decision between Diagnostics WG and Release WG. Some of the
criteria considered might be:
* If the tool fits into a key category as listed below.
* Whether the tool is actively used by the ecosystem.
* The availability of alternatives.
* Impact to the overall ecosystem if the tool is not working.
* The availability of reliable test suite that can be integrated into our CI.
* The availability of maintainer or community collaborator who will help
resolve issues when there are CI failures.
* If the tool is maintained by the Node.js Foundation GitHub organization.
The current categories of tools/APIs that fall under these Tiers are:
* FFDC (F) - First failure data capture, easy to consume initial diagnostic
information.
* Tracing (T) - use of logging to provide information about execution flow.
* Memory (M) - tools that provide additional information about memory
used in the Heap or by native code.
* Profiling (P) - tools that provide additional information about where
CPU cycles are being spent.
* AsyncFlow (A) - tools that provide additional insight into asynchronous
execution flow.
## Adding a tool to this list
Any tool that might be used to investigate issues when running Node.js can
be added to the list. If there is a new tool that should be added to the
list, it should start by being added to the "Not yet classified" or
"Tier 4" lists. Once it has been added to the list "promotion" to Tier 3
through Tier 1 requires that the requirements listed above be met AND
have agreement from Diagnostics WG and Release WG based on the criteria
listed above.
## Tiers
The tools are currently assigned to Tiers as follows:
## Tier 1
| Tool Type | Tool/API Name | Regular Testing in Node.js CI | Integrated with Node.js | Target Tier |
| --------- | --------------------- | ----------------------------- | ----------------------- | ----------- |
| FFDC | [diagnostic report][] | Yes | Yes | 1 |
| | | | | |
## Tier 2
| Tool Type | Tool/API Name | Regular Testing in Node.js CI | Integrated with Node.js | Target Tier |
| --------- | ---------------------------- | ----------------------------- | ----------------------- | ----------- |
| Debugger | [Chrome DevTools Protocol][] | Yes | Yes | 1 |
## Tier 3
| Tool Type | Tool/API Name | Regular Testing in Node.js CI | Integrated with Node.js | Target Tier |
| --------- | ------------------------------------ | ----------------------------- | ----------------------- | ----------- |
| Profiling | V8 CPU profiler | Partial (V8 Tests) | Yes | 1 |
| Profiling | --prof/--prof-process flags | Yes | Yes | 1 |
| Profiling | V8 CodeEventHandler API | Partial (V8 Tests) | Yes | 2 |
| Profiling | V8 --interpreted-frames-native-stack | Yes | Yes | 2 |
| Profiling | [Linux perf][] | Yes | Partial | 2 |
| Profiling | [node-clinic][] | No | No | 3 |
| Debugger | [Chrome DevTools Frontend][] | No | No | 3 |
## Tier 4
| Tool Type | Tool/API Name | Regular Testing in Node.js CI | Integrated with Node.js | Target Tier |
| --------- | ------------- | ----------------------------- | ----------------------- | ----------- |
| Profiling | [0x][] | No | No | 3 |
## Not yet classified
| Tool Type | Tool/API Name | Regular Testing in Node.js CI | Integrated with Node.js | Target Tier |
| --------- | ----------------------------------------- | ----------------------------- | ----------------------- | ----------- |
| Memory | V8 heap profiler | No | Yes | 1 |
| Memory | V8 sampling heap profiler | No | Yes | 1 |
| AsyncFlow | [Async Hooks (API)][] | ? | Yes | 1 |
| Debugger | [Command line Debug Client][] | ? | Yes | 1 |
| Tracing | [trace\_events (API)][trace_events (API)] | No | Yes | 1 |
| Tracing | trace\_gc | No | Yes | 1 |
[0x]: https://github.com/davidmarkclements/0x
[Async Hooks (API)]: https://nodejs.org/api/async_hooks.html
[Chrome DevTools Frontend]: https://developer.chrome.com/docs/devtools/
[Chrome DevTools Protocol]: https://chromedevtools.github.io/devtools-protocol/
[Command line Debug Client]: https://nodejs.org/api/debugger.html
[Linux perf]: https://perf.wiki.kernel.org/index.php/Main_Page
[diagnostic report]: https://nodejs.org/api/report.html
[node-clinic]: https://github.com/clinicjs/node-clinic/
[trace_events (API)]: https://nodejs.org/api/tracing.html

View File

@ -0,0 +1,27 @@
# Node.js Distribution Policy
This document describes some policies around what is and is not included in the
Node.js distribution.
## Inclusion
The Node.js distribution includes some external software that the Node.js
project does not maintain. The choice to include a particular piece of software
should not imply anything about that software relative to its competitors; in
some cases, software was added when it had no competitors. While the Node.js
project supports and encourages competition in the JavaScript ecosystem, as a
policy, the Node.js project does not include multiple dependencies or tools that
serve the same purpose.
The following user-accessible external tools or libraries are the ones chosen
for their particular purposes:
* JavaScript engine: V8
* Package manager: `npm`
* Package manager version manager: Corepack
Being user-accessible, removal or replacement of these projects could happen
only as a semver-major change, unless the related feature or project is
documented as experimental. In addition, Node.js includes external projects as
internal dependencies. These may be replaced or removed at any time, provided
that doing so is not a breaking change.

Binary file not shown.

After

Width:  |  Height:  |  Size: 260 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 167 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 46 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 17 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 15 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 20 KiB

View File

@ -0,0 +1,89 @@
# Feature request management
Feature requests are a valuable source of input to the project.
They help our maintainers understand what additions will be of
value to users of the Node.js runtime.
At the same time, the project is volunteer run and does not
have the ability to direct resources toward specific work. The
features which are implemented are those for which volunteers
are individually motivated to work on. The best way to ensure
a feature gets implemented is to create a PR to add it.
The project strives to support people who do that.
An open feature request does not provide any indication that work
on that feature will take place and after some period of time
may be detrimental as it may result in an expectation that will
never be fulfilled.
This process tries to balance retaining the valuable input
we get through feature requests and the overhead of
maintaining open feature requests that will never get
implemented.
## Creating feature requests
A feature request can be created by adding the `feature request`
label to an issue. This may be done automatically when the issue
is opened or at a later point (often when investigation of a bug
report results in it being considered a feature request as opposed
to a bug).
The current list of feature requests can be found through the
[is:issue is:open label:"feature request"](https://github.com/nodejs/node/issues?q=is%3Aissue+is%3Aopen+label%3A%22feature+request%22)
query.
## Triage of feature requests
There is no set process for triaging/handling feature requests.
Individual collaborators review issues marked as `feature request`
along with other issues and may or may not decide to
work on an implementation or advocate on their behalf.
If a collaborator believes a feature request must be implemented
they can add the `never-stale` label to the issue and it will
be excluded from the automated feature request handling
as outlined below.
## Expressing support for a feature request
If you come across a feature request and want to add your
support for that feature please express your support
with the thumbs up emoji as a reaction. At some point in the
future we may use this as additional input in the automated
handling of feature requests.
## Automated feature request handling
Our experience is that most feature requests that are
likely to be addressed, will be addressed within the first
6 months after they are submitted.
Once there has been no activity on a feature request for
5 months, the following comment will be added
to the issue:
```markdown
There has been no activity on this feature request for
5 months and it is unlikely to be implemented.
It will be closed 6 months after the last non-automated comment.
For more information on how the project manages
feature requests, please consult the
[feature request management document](https://github.com/nodejs/node/blob/HEAD/doc/contributing/feature-request-management.md).
```
If there is no additional activity/discussion on the
feature request in the next month, the following
comment is added to the issue and the issue will be
closed:
```markdown
There has been no activity on this feature request
and it is being closed. If you feel closing this issue is not the
right thing to do, please leave a comment.
For more information on how the project manages
feature requests, please consult the
[feature request management document](https://github.com/nodejs/node/blob/HEAD/doc/contributing/feature-request-management.md).
```

View File

@ -0,0 +1,134 @@
# GN Build
Similar to GYP, GN is a build system designed for building Chromium. The
official builds of Node.js are built with GYP, but GN build files are also
provided as an unofficial alternative build system.
The GN build files only support a subset of the Node.js build configurations.
It's not required for all pull requests to Node.js to support GN builds, and the
CI of Node.js does not currently test GN builds, though Node.js welcomes pull
requests that improve GN support or fix breakages introduced by other pull
requests.
Currently the GN build is used by:
1. Electron for building Node.js together with Chromium.
2. V8 for testing the integration of Node.js in CI.
## Files for GN support
Node.js contains following GN build files:
* `node.gni` - Public GN arguments for configuring the build.
* `*/BUILD.gn` - This is the file where GN looks for build rules, in Node.js it
just includes the `unofficial.gni` file to avoid accidental modifications.
* `*/unofficial.gni` - The actual build rules for each component.
## Build with GN
Unlike GYP, the GN tool does not include any built-in rules for compiling a
project, which means projects building with GN must provide their own build
configurations for things like how to invoke a C++ compiler.
Chromium related projects like V8 and skia choose to reuse Chromium's build
configurations, and V8's Node.js integration testing repository
[`node-ci`][node-ci] can be reused for building Node.js.
### 1. Install `depot_tools`
You'll need to install [`depot_tools`][depot-tools] the toolset
used for fetching Chromium and its dependencies.
```bash
git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git
export PATH=/path/to/depot_tools:$PATH
```
You can ensure `depot_tools` is correctly added to your PATH by running
`which gn` and confirming that it returns `/path/to/depot_tools/gn`.
**NOTE:** On Windows you'll also need to set the environment variable
`DEPOT_TOOLS_WIN_TOOLCHAIN=0`. To do so, open `Control Panel` → `System and
Security` → `System` → `Advanced system settings` and add a system variable
`DEPOT_TOOLS_WIN_TOOLCHAIN` with value `0`. This tells `depot_tools` to use
your locally installed version of Visual Studio (by default, `depot_tools` will
try to download a Google-internal version that only Googlers have access to).
### 2. Checkout Node.js Source Code
To check out the latest main branch of Node.js for building, use the `fetch`
tool from `depot_tools`:
```bash
mkdir node_gn
cd node_gn
fetch node
```
You can choose to save some time by omitting git history:
```bash
fetch --no-history node
```
After syncing is done, you will get a directory structure like this:
```console
node_gn/
├── .gclient
├── .gclient_entries
├── ...
└── node
├── DEPS
├── ...
├── build/
├── node/
└── tools/
```
The `node_gn` is a workspace directory, which only contains configurations and
caches of the `gclient` tool from `depot_tools`, and the repository is checked
out at `node_gn/node`.
The `node_gn/node` directory is not a checkout of Node.js, it is actually
[node-ci](https://chromium.googlesource.com/v8/node-ci/), the repository used by
V8 for testing integration with Node.js. The source code of Node.js is checked
out at `node_gn/node/node`.
### 3. Build
GN only supports [`ninja`](https://ninja-build.org) for building. To build
Node.js with GN you'll first need to generate `ninja` build files and then invoke
`ninja` to perform the build.
The `node-ci` repository provides a script for calling GN:
```bash
cd node # Enter `node_gn/node` which contains a node-ci checkout
./tools/gn-gen.py out/Release
```
which writes `ninja` build files into the `out/Release` directory under
`node_gn/node`. To see all possible configurable options, run
`tools/gn-gen.py --help`.
When `gn-gen.py` has executed successfully, you can then execute `ninja`:
```bash
ninja -C out/Release node
```
After the build is completed, the compiled Node.js executable can be found in
`out/Release/node`.
## Status of the GN build
Currently the GN build of Node.js is not fully functioning. Some tests
are still failing with the GN build, and there may be other small pitfall
for certain configuration options.
An effort is currently underway to make GN build work without using `depot_tools`,
which is tracked in [#51689](https://github.com/nodejs/node/issues/51689).
[depot-tools]: https://commondatastorage.googleapis.com/chrome-infra-docs/flat/depot_tools/docs/html/depot_tools_tutorial.html#_setting_up
[node-ci]: https://chromium.googlesource.com/v8/node-ci

View File

@ -0,0 +1,45 @@
# Node.js Core Development Flags
These flags are specifically designed for use in Node.js core development and are not intended for general
application usage.
> \[!NOTE]
> These APIs are not bound by semantic versioning rules, and they can be altered or removed in any version of Node.js
## Command Line Interface (CLI)
### Flags
#### `--debug-arraybuffer-allocations`
Enables debugging of `ArrayBuffer` allocations.
#### `--experimental-quic`
Enable QUIC Protocol (under development)
#### `--expose-internals`
Allows the usage of `internal/*` modules, granting access to internal Node.js functionality.
#### `--inspect-brk-node[=[host:]port]`
Pauses execution at the start of Node.js application code, waiting for a debugger to connect on the specified
`host` and `port`. This is useful for debugging application startup issues. If `host` and `port` are not
provided, it defaults to `127.0.0.1:9229`.
#### `--node-snapshot`
Enables the use of Node.js snapshots, potentially improving startup performance.
#### `--test-udp-no-try-send`
Used for testing UDP functionality without attempting to send data.
#### `--trace-promises`
Enables tracing of promises for debugging and performance analysis.
#### `--verify-base-objects`
Allows verification of base objects for debugging purposes.

View File

@ -0,0 +1,558 @@
# Investigating memory leaks with Valgrind
A Node.js process may run out of memory due to excessive consumption of
native memory. Native Memory is memory which is not managed by the
V8 Garbage collector and is allocated either by the Node.js runtime, its
dependencies or native [addons](https://nodejs.org/docs/latest/api/n-api.html).
This guide provides information on how to use Valgrind to investigate these
issues on Linux platforms.
## Valgrind
[Valgrind](https://valgrind.org/docs/manual/quick-start.html) is a
tool available on Linux distributions which can be used to investigate
memory usage including identifying memory leaks (memory which is
allocated and not freed) and other memory related problems
like double freeing memory.
To use Valgrind:
* Be patient, running under Valgrind slows execution significantly
due to the checks being performed.
* Reduce your test case to the smallest reproduce. Due to the slowdown it is
important to run the minimum test case in order to be able to do it in
a reasonable time.
## Installation
It is an optional package in most cases and must be installed explicitly.
For example on Debian/Ubuntu:
```bash
apt-get install valgrind
```
## Invocation
The simplest invocation of Valgrind is:
```bash
valgrind node test.js
```
with the output being:
```console
user1@minikube1:~/valgrind/node-addon-examples/1_hello_world/napi$ valgrind node test.js
==28993== Memcheck, a memory error detector
==28993== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==28993== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==28993== Command: node test.js
==28993==
==28993== Use of uninitialised value of size 8
==28993== at 0x12F2279: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==28993== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==28993== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==28993== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==28993== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==28993== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==28993== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==28993== by 0x12F3E9C: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==28993== by 0x12F3C77: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==28993== by 0xC7C9CF: v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==28993== by 0xC7CE87: v8::internal::Execution::Call(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, int, v8::internal::Handle<v8::internal::Object>*) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==28993== by 0xB4CF3A: v8::Function::Call(v8::Local<v8::Context>, v8::Local<v8::Value>, int, v8::Local<v8::Value>*) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==28993==
--28993-- WARNING: unhandled amd64-linux syscall: 332
--28993-- You may be able to write your own handler.
--28993-- Read the file README_MISSING_SYSCALL_OR_IOCTL.
--28993-- Nevertheless we consider this a bug. Please report
--28993-- it at http://valgrind.org/support/bug_reports.html.
==28993==
==28993== HEAP SUMMARY:
==28993== in use at exit: 6,140 bytes in 23 blocks
==28993== total heap usage: 12,888 allocs, 12,865 frees, 13,033,244 bytes allocated
==28993==
==28993== LEAK SUMMARY:
==28993== definitely lost: 0 bytes in 0 blocks
==28993== indirectly lost: 0 bytes in 0 blocks
==28993== possibly lost: 304 bytes in 1 blocks
==28993== still reachable: 5,836 bytes in 22 blocks
==28993== suppressed: 0 bytes in 0 blocks
==28993== Rerun with --leak-check=full to see details of leaked memory
==28993==
==28993== For counts of detected and suppressed errors, rerun with: -v
==28993== Use --track-origins=yes to see where uninitialised values come
```
This reports that Node.js is not _completely_ clean as there is some memory
that was allocated but not freed when the process shut down. It is often
impractical/not worth being completely clean in this respect. Modern
operating systems will clean up the memory of the process after the
shutdown while attempting to free all memory to get a clean
report may have a negative impact on the code complexity and
shutdown times. Node.js does a pretty good job only leaving on
the order of 6 KB that are not freed on shutdown.
## An obvious memory leak
Leaks can be introduced in native addons and the following is a simple
example leak based on the "Hello world" addon from
[node-addon-examples](https://github.com/nodejs/node-addon-examples).
In this example, a loop which allocates approximately 1 MiB of memory and never
frees it has been added:
```cpp
void* malloc_holder = nullptr;
napi_value Method(napi_env env, napi_callback_info info) {
napi_status status;
napi_value world;
status = napi_create_string_utf8(env, "world", 5, &world);
assert(status == napi_ok);
// NEW LEAK HERE
for (int i=0; i < 1024; i++) {
malloc_holder = malloc(1024);
}
return world;
}
```
When trying to create a memory leak you need to ensure that
the compiler has not optimized out the code that creates
the leak. For example, by assigning the result of the allocation
to either a global variable or a variable that will be read
afterwards the compiler will not optimize it out along with
the malloc and Valgrind will properly report the memory leak.
If `malloc_holder` in the example above is made into a
local variable then the compiler may freely remove
it along with the allocations (since it is not used)
and Valgrind will not find any leaks since they
will no longer exist in the code being run.
Running Valgrind on this code shows the following:
```console
user1@minikube1:~/valgrind/node-addon-examples/1_hello_world/napi$ valgrind node hello.js
==1504== Memcheck, a memory error detector
==1504== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==1504== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==1504== Command: node hello.js
==1504==
==1504== Use of uninitialised value of size 8
==1504== at 0x12F2279: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==1504== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==1504== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==1504== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==1504== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==1504== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==1504== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==1504== by 0x12F3E9C: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==1504== by 0x12F3C77: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==1504== by 0xC7C9CF: v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==1504== by 0xC7CE87: v8::internal::Execution::Call(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, int, v8::internal::Handle<v8::internal::Object>*) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==1504== by 0xB4CF3A: v8::Function::Call(v8::Local<v8::Context>, v8::Local<v8::Value>, int, v8::Local<v8::Value>*) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==1504==
--1504-- WARNING: unhandled amd64-linux syscall: 332
--1504-- You may be able to write your own handler.
--1504-- Read the file README_MISSING_SYSCALL_OR_IOCTL.
--1504-- Nevertheless we consider this a bug. Please report
--1504-- it at http://valgrind.org/support/bug_reports.html.
world
==1504==
==1504== HEAP SUMMARY:
==1504== in use at exit: 1,008,003 bytes in 1,032 blocks
==1504== total heap usage: 17,603 allocs, 16,571 frees, 18,306,103 bytes allocated
==1504==
==1504== LEAK SUMMARY:
==1504== definitely lost: 996,064 bytes in 997 blocks
==1504== indirectly lost: 0 bytes in 0 blocks
==1504== possibly lost: 3,304 bytes in 4 blocks
==1504== still reachable: 8,635 bytes in 31 blocks
==1504== of which reachable via heuristic:
==1504== multipleinheritance: 48 bytes in 1 blocks
==1504== suppressed: 0 bytes in 0 blocks
==1504== Rerun with --leak-check=full to see details of leaked memory
==1504==
==1504== For counts of detected and suppressed errors, rerun with: -v
==1504== Use --track-origins=yes to see where uninitialised values come from
==1504== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
```
Valgrind is reporting a problem as it shows 996,604 bytes as
definitely lost and the question is how to find where that memory was
allocated. The next step is to rerun as suggested in the
output with `--leak-check=full`:
```bash
user1@minikube1:~/valgrind/node-addon-examples/1_hello_world/napi$ valgrind --leak-check=full node hello.js
==4174== Memcheck, a memory error detector
==4174== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==4174== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==4174== Command: node hello.js
==4174==
==4174== Use of uninitialised value of size 8
==4174== at 0x12F2279: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x12F3E9C: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x12F3C77: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0xC7C9CF: v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0xC7CE87: v8::internal::Execution::Call(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, int, v8::internal::Handle<v8::internal::Object>*) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0xB4CF3A: v8::Function::Call(v8::Local<v8::Context>, v8::Local<v8::Value>, int, v8::Local<v8::Value>*) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174==
--4174-- WARNING: unhandled amd64-linux syscall: 332
--4174-- You may be able to write your own handler.
--4174-- Read the file README_MISSING_SYSCALL_OR_IOCTL.
--4174-- Nevertheless we consider this a bug. Please report
--4174-- it at http://valgrind.org/support/bug_reports.html.
world
==4174==
==4174== HEAP SUMMARY:
==4174== in use at exit: 1,008,003 bytes in 1,032 blocks
==4174== total heap usage: 17,606 allocs, 16,574 frees, 18,305,977 bytes allocated
==4174==
==4174== 64 bytes in 1 blocks are definitely lost in loss record 17 of 35
==4174== at 0x4C3017F: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4174== by 0x9AEAD5: napi_module_register (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x4010732: call_init (dl-init.c:72)
==4174== by 0x4010732: _dl_init (dl-init.c:119)
==4174== by 0x40151FE: dl_open_worker (dl-open.c:522)
==4174== by 0x5D052DE: _dl_catch_exception (dl-error-skeleton.c:196)
==4174== by 0x40147C9: _dl_open (dl-open.c:605)
==4174== by 0x4E3CF95: dlopen_doit (dlopen.c:66)
==4174== by 0x5D052DE: _dl_catch_exception (dl-error-skeleton.c:196)
==4174== by 0x5D0536E: _dl_catch_error (dl-error-skeleton.c:215)
==4174== by 0x4E3D734: _dlerror_run (dlerror.c:162)
==4174== by 0x4E3D050: dlopen@@GLIBC_2.2.5 (dlopen.c:87)
==4174== by 0x9B29A0: node::binding::DLOpen(v8::FunctionCallbackInfo<v8::Value> const&)::{lambda(node::binding::DLib*)#1}::operator()(node::binding::DLib*) const (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174==
==4174== 304 bytes in 1 blocks are possibly lost in loss record 27 of 35
==4174== at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4174== by 0x40134A6: allocate_dtv (dl-tls.c:286)
==4174== by 0x40134A6: _dl_allocate_tls (dl-tls.c:530)
==4174== by 0x5987227: allocate_stack (allocatestack.c:627)
==4174== by 0x5987227: pthread_create@@GLIBC_2.2.5 (pthread_create.c:644)
==4174== by 0xAAF9DC: node::inspector::Agent::Start(std::string const&, node::DebugOptions const&, std::shared_ptr<node::HostPort>, bool) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x9A8BE7: node::Environment::InitializeInspector(std::unique_ptr<node::inspector::ParentInspectorHandle, std::default_delete<node::inspector::ParentInspectorHandle> >) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0xA1C9A5: node::NodeMainInstance::CreateMainEnvironment(int*) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0xA1CB42: node::NodeMainInstance::Run() (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x9ACB67: node::Start(int, char**) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x5BBFB96: (below main) (libc-start.c:310)
==4174==
==4174== 2,000 bytes in 2 blocks are possibly lost in loss record 33 of 35
==4174== at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4174== by 0x9794979: Method(napi_env__*, napi_callback_info__*) (in /home/user1/valgrind/node-addon-examples/1_hello_world/napi/build/Release/hello.node)
==4174== by 0x98F764: v8impl::(anonymous namespace)::FunctionCallbackWrapper::Invoke(v8::FunctionCallbackInfo<v8::Value> const&) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0xBA6FC8: v8::internal::MaybeHandle<v8::internal::Object> v8::internal::(anonymous namespace)::HandleApiCallHelper<false>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::FunctionTemplateInfo>, v8::internal::Handle<v8::internal::Object>, v8::internal::BuiltinArguments) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0xBA8DB6: v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x1376358: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174==
==4174== 997,000 bytes in 997 blocks are definitely lost in loss record 35 of 35
==4174== at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4174== by 0x9794979: Method(napi_env__*, napi_callback_info__*) (in /home/user1/valgrind/node-addon-examples/1_hello_world/napi/build/Release/hello.node)
==4174== by 0x98F764: v8impl::(anonymous namespace)::FunctionCallbackWrapper::Invoke(v8::FunctionCallbackInfo<v8::Value> const&) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0xBA6FC8: v8::internal::MaybeHandle<v8::internal::Object> v8::internal::(anonymous namespace)::HandleApiCallHelper<false>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::FunctionTemplateInfo>, v8::internal::Handle<v8::internal::Object>, v8::internal::BuiltinArguments) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0xBA8DB6: v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x1376358: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174==
==4174== LEAK SUMMARY:
==4174== definitely lost: 997,064 bytes in 998 blocks
==4174== indirectly lost: 0 bytes in 0 blocks
==4174== possibly lost: 2,304 bytes in 3 blocks
==4174== still reachable: 8,635 bytes in 31 blocks
==4174== of which reachable via heuristic:
==4174== multipleinheritance: 48 bytes in 1 blocks
==4174== suppressed: 0 bytes in 0 blocks
==4174== Reachable blocks (those to which a pointer was found) are not shown.
==4174== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==4174==
==4174== For counts of detected and suppressed errors, rerun with: -v
==4174== Use --track-origins=yes to see where uninitialised values come from
==4174== ERROR SUMMARY: 5 errors from 5 contexts (suppressed: 0 from 0)
```
This is the most interesting part of the report:
```console
==4174== 997,000 bytes in 997 blocks are definitely lost in loss record 35 of 35
==4174== at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4174== by 0x9794979: Method(napi_env__*, napi_callback_info__*) (in /home/user1/valgrind/node-addon-examples/1_hello_world/napi/build/Release/hello.node)
==4174== by 0x98F764: v8impl::(anonymous namespace)::FunctionCallbackWrapper::Invoke(v8::FunctionCallbackInfo<v8::Value> const&) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0xBA6FC8: v8::internal::MaybeHandle<v8::internal::Object> v8::internal::(anonymous namespace)::HandleApiCallHelper<false>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::FunctionTemplateInfo>, v8::internal::Handle<v8::internal::Object>, v8::internal::BuiltinArguments) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0xBA8DB6: v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x1376358: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
```
From the stack trace we can tell that the leak came from a native addon:
```console
==4174== by 0x9794979: Method(napi_env__*, napi_callback_info__*) (in /home/user1/valgrind/node-addon-examples/1_hello_world/napi/build/Release/hello.node)
```
What we can't tell is where in the native addon the memory is being
allocated. This is because by default the addon is compiled without
the debug symbols which Valgrind needs to be able to provide more
information.
It is possible to hide leaks related to Node.js itself in future Valgrind runs
using the suppression feature of Valgrind.
## Generating a Valgrind suppression file
Valgrind uses suppression files to hide issues found from the summary. Generate
a log file with embedded suppressions using the `--gen-suppressions` and
`--log-file` flags:
```bash
valgrind --leak-check=full \
--gen-suppressions=all \
--log-file=./valgrind-out.txt \
node hello.js
```
Valgrind will save the output to the log file specified. After each heap in the
summary, Valgrind will include a suppression record: a structure that Valgrind
can use to ignore specific memory issues. Suppression records can be saved to a
suppression file which Valgrind can use in subsequent executions to hide various
memory errors. This is an example of the suppression records from the previous
call:
```text
{
<insert_a_suppression_name_here>
Memcheck:Value8
obj:/home/kevin/.nvm/versions/node/v12.14.1/bin/node
obj:/home/kevin/.nvm/versions/node/v12.14.1/bin/node
obj:/home/kevin/.nvm/versions/node/v12.14.1/bin/node
obj:/home/kevin/.nvm/versions/node/v12.14.1/bin/node
obj:/home/kevin/.nvm/versions/node/v12.14.1/bin/node
obj:/home/kevin/.nvm/versions/node/v12.14.1/bin/node
obj:/home/kevin/.nvm/versions/node/v12.14.1/bin/node
obj:/home/kevin/.nvm/versions/node/v12.14.1/bin/node
obj:/home/kevin/.nvm/versions/node/v12.14.1/bin/node
fun:_ZN2v88internal12_GLOBAL__N_16InvokeEPNS0_7IsolateERKNS1_12InvokeParamsE
fun:_ZN2v88internal9Execution4CallEPNS0_7IsolateENS0_6HandleINS0_6ObjectEEES6_iPS6_
fun:_ZN2v88Function4CallENS_5LocalINS_7ContextEEENS1_INS_5ValueEEEiPS5_
}
{
<insert_a_suppression_name_here>
Memcheck:Leak
match-leak-kinds: definite
fun:_Znwm
fun:napi_module_register
fun:call_init.part.0
fun:call_init
fun:_dl_init
fun:_dl_catch_exception
fun:dl_open_worker
fun:_dl_catch_exception
fun:_dl_open
fun:dlopen_doit
fun:_dl_catch_exception
fun:_dl_catch_error
fun:_dlerror_run
}
{
<insert_a_suppression_name_here>
Memcheck:Leak
match-leak-kinds: possible
fun:calloc
fun:allocate_dtv
fun:_dl_allocate_tls
fun:allocate_stack
fun:pthread_create@@GLIBC_2.2.5
fun:_ZN4node9inspector5Agent5StartERKSsRKNS_12DebugOptionsESt10shared_ptrINS_8HostPortEEb
fun:_ZN4node11Environment19InitializeInspectorESt10unique_ptrINS_9inspector21ParentInspectorHandleESt14default_deleteIS3_EE
fun:_ZN4node16NodeMainInstance21CreateMainEnvironmentEPi
fun:_ZN4node16NodeMainInstance3RunEv
fun:_ZN4node5StartEiPPc
fun:(below main)
}
```
Create a file (eg. `node-12.14.1.supp`) with the contents of the suppression
records, and run Valgrind with the suppression file previously created:
```bash
valgrind --leak-check=full \
--suppressions=./node-12.14.1.supp \
node hello.js
```
Now, the Valgrind leak summary for suppressed issues are only mentioned as
`suppressed` in the leak summary:
```console
==12471== HEAP SUMMARY:
==12471== in use at exit: 8,067 bytes in 31 blocks
==12471== total heap usage: 16,482 allocs, 16,451 frees, 17,255,689 bytes allocated
==12471==
==12471== LEAK SUMMARY:
==12471== definitely lost: 0 bytes in 0 blocks
==12471== indirectly lost: 0 bytes in 0 blocks
==12471== possibly lost: 0 bytes in 0 blocks
==12471== still reachable: 7,699 bytes in 29 blocks
==12471== of which reachable via heuristic:
==12471== multipleinheritance: 48 bytes in 1 blocks
==12471== suppressed: 368 bytes in 2 blocks
```
## Enabling debug symbols to get more information
Leaks may be either in addons or Node.js itself. The sections which
follow cover the steps needed to enable debug symbols to get more info.
### Native addons
To enable debug symbols for all of your addons that are compiled on
install use:
```bash
npm install --debug
```
Any options which are not consumed by npm are passed on to node-gyp and this
results in the addons being compiled with the debug option.
If the native addon contains pre-built binaries you will need to force
a rebuild.
```bash
npm install --debug
npm rebuild
```
The next step is to run Valgrind after the rebuild. This time the information
for the leaking location includes the name of the source file and the
line number:
```console
==18481== 997,000 bytes in 997 blocks are definitely lost in loss record 35 of 35
==18481== at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
>>>>> ==18481== by 0x9794989: Method(napi_env__*, napi_callback_info__*) (hello.cc:13) <<<<<
==18481== by 0x98F764: v8impl::(anonymous namespace)::FunctionCallbackWrapper::Invoke(v8::FunctionCallbackInfo<v8::Value> const&) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==18481== by 0xBA6FC8: v8::internal::MaybeHandle<v8::internal::Object> v8::internal::(anonymous namespace)::HandleApiCallHelper<false>(v8::internal:: Isolate*, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::FunctionTemplateInfo>, v8::internal::Handle<v8::internal::Object>, v8::internal::BuiltinArguments) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==18481== by 0xBA8DB6: v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==18481== by 0x1376358: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==18481== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==18481== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==18481== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==18481== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==18481== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==18481== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
```
This new output shows us exactly where the leak is occurring in the file `hello.cc`:
```cpp
6 void* malloc_holder = nullptr;
7 napi_value Method(napi_env env, napi_callback_info info) {
8 napi_status status;
9 napi_value world;
10 status = napi_create_string_utf8(env, "world", 5, &world);
11 assert(status == napi_ok);
12 for (int i=0; i< 1000; i++) {
13 malloc_holder = malloc(1000); // <<<<<< This is where we are allocating the memory that is not freed
14 }
15 return world;
16 }
```
### Node.js binary
If the leak is not in an addon and is instead in the Node.js binary itself,
you may need to compile node yourself and turn on debug symbols. Looking at
this entry reported by Valgrind, with a release binary we see:
```console
==4174== 304 bytes in 1 blocks are possibly lost in loss record 27 of 35
==4174== at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4174== by 0x40134A6: allocate_dtv (dl-tls.c:286)
==4174== by 0x40134A6: _dl_allocate_tls (dl-tls.c:530)
==4174== by 0x5987227: allocate_stack (allocatestack.c:627)
==4174== by 0x5987227: pthread_create@@GLIBC_2.2.5 (pthread_create.c:644)
==4174== by 0xAAF9DC: node::inspector::Agent::Start(std::string const&, node::DebugOptions const&, std::shared_ptr<node::HostPort>, bool) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x9A8BE7: node::Environment::InitializeInspector(std::unique_ptr<node::inspector::ParentInspectorHandle, std::default_delete<node::inspector::ParentInspectorHandle> >) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0xA1C9A5: node::NodeMainInstance::CreateMainEnvironment(int*) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0xA1CB42: node::NodeMainInstance::Run() (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x9ACB67: node::Start(int, char**) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node)
==4174== by 0x5BBFB96: (below main) (libc-start.c:310)
```
This gives us some information of where to look (`node::inspector::Agent::Start`)
but not where in that function. We get more information than you might expect
(or see by default with addons) because the Node.js binary exports many of
its symbols using `-rdynamic` so that they can be used by addons. If the stack
gives you enough information to track down where the leak is, that's great,
otherwise the next step is to compile a debug build of Node.js.
To get additional information with Valgrind:
* Check out the Node.js source corresponding to the release that you
want to debug. For example:
```bash
git clone https://github.com/nodejs/node.git
git checkout v12.14.1
```
* Compile with debug enabled (for additional info see
[building a debug build](https://github.com/nodejs/node/blob/v12.14.1/BUILDING.md#building-a-debug-build)).
For example, on \*nix:
```bash
./configure --debug
make -j4
```
* Make sure to run with your compiled debug version of Node.js. Having used
`./configure --debug`, two binaries will have been built when `make` was run.
You must use the one which is in `out/Debug`.
Running Valgrind using the debug build of Node.js shows:
```console
==44112== 592 bytes in 1 blocks are possibly lost in loss record 26 of 27
==44112== at 0x4C2BF79: calloc (vg_replace_malloc.c:762)
==44112== by 0x4012754: _dl_allocate_tls (in /usr/lib64/ld-2.17.so)
==44112== by 0x586287B: pthread_create@@GLIBC_2.2.5 (in /usr/lib64/libpthread-2.17.so)
==44112== by 0xFAB2D2: node::inspector::(anonymous namespace)::StartDebugSignalHandler() (inspector_agent.cc:140)
==44112== by 0xFACB10: node::inspector::Agent::Start(std::string const&, node::DebugOptions const&, std::shared_ptr<node::HostPort>, bool) (inspector_agent.cc:777)
==44112== by 0xE3A0BB: node::Environment::InitializeInspector(std::unique_ptr<node::inspector::ParentInspectorHandle, std::default_delete<node::inspector::ParentInspectorHandle> >) (node.cc:216)
==44112== by 0xEE8F3E: node::NodeMainInstance::CreateMainEnvironment(int*) (node_main_instance.cc:222)
==44112== by 0xEE8831: node::NodeMainInstance::Run() (node_main_instance.cc:108)
==44112== by 0xE3CDEC: node::Start(int, char**) (node.cc:996)
==44112== by 0x22D8BBF: main (node_main.cc:126)
```
Now we can see the specific file name and line in the Node.js code which
caused the allocation (inspector\_agent.cc:140).
We can examine that line (and its surrounding code) to
find a solution for the memory leak.

View File

@ -0,0 +1,76 @@
# Issues
* [Asking for general help](#asking-for-general-help)
* [Discussing non-technical topics](#discussing-non-technical-topics)
* [Submitting a bug report](#submitting-a-bug-report)
* [Triaging a bug report](#triaging-a-bug-report)
## Asking for general help
Because the level of activity in the `nodejs/node` repository is so high,
questions or requests for general help using Node.js should be directed at
the [Node.js help repository][].
## Discussing non-technical topics
Discussion of non-technical topics (such as intellectual property and trademark)
should be directed to the [Technical Steering Committee (TSC) repository][].
## Submitting a bug report
When opening a new issue in the `nodejs/node` issue tracker, users will be
presented with a choice of issue templates. If you believe that you have
uncovered a bug in Node.js, please fill out the `Bug Report` template to the
best of your ability. Do not worry if you cannot answer every detail; just fill
in what you can.
The two most important pieces of information we need in order to properly
evaluate the report is a description of the behavior you are seeing and a simple
test case we can use to recreate the problem on our own. If we cannot recreate
the issue, it becomes impossible for us to fix.
In order to rule out the possibility of bugs introduced by userland code, test
cases should be limited, as much as possible, to using _only_ Node.js APIs.
If the bug occurs only when you're using a specific userland module, there is
a very good chance that either (a) the module has a bug or (b) something in
Node.js changed that broke the module.
See [How to create a Minimal, Complete, and Verifiable example](https://stackoverflow.com/help/mcve).
## Triaging a bug report
Once an issue has been opened, it is common for there to be discussion
around it. Some contributors may have differing opinions about the issue,
including whether the behavior being seen is a bug or a feature. This discussion
is part of the process and should be kept focused, helpful, and professional.
The objective of helping with triaging issues (in core and help repos) is to
help reduce the issue backlog and keep the issue tracker healthy, while enabling
newcomers another meaningful way to get engaged and contribute.
Anyone with a reasonable understanding of Node.js programming and the
project's GitHub organization plus a few contributions to the project
(commenting on issues or PRs) can apply for and become a triager. Open a PR
on the README.md of this project with: i) a request to be added as a triager,
ii) the motivation for becoming a triager, and iii) agreement on reading,
understanding, and adhering to the project's [Code Of Conduct](https://github.com/nodejs/admin/blob/HEAD/CODE_OF_CONDUCT.md).
The triage role enables the ability to carry out the most common triage
activities, such as applying labels and closing/reopening/assigning issues.
For more information on the roles and permissions, see ["Permission levels for
repositories owned by an organization"](https://docs.github.com/en/github/setting-up-and-managing-organizations-and-teams/repository-permission-levels-for-an-organization#permission-levels-for-repositories-owned-by-an-organization).
When triagging issues and PRs:
* Show patience and empathy, especially to first-time contributors.
* Show no patience towards spam or troll, close the issue without interacting with it and
report the user to the moderation repository.
* If you're not able to reproduce an issue, leave a comment asking for more info and
add the `needs more info` label.
* Ideally issues should be closed only when they have been fixed or answered (and
merged for pull requests). Closing an issue (or PR) earlier can be seen as
dismissive from the point of view of the reporter/author.
Always try to communicate the reason for closing the issue/PR.
[Node.js help repository]: https://github.com/nodejs/help/issues
[Technical Steering Committee (TSC) repository]: https://github.com/nodejs/TSC/issues

View File

@ -0,0 +1,391 @@
# Maintaining V8 in Node.js
## Background
V8 follows the Chromium release schedule. The support horizon for Chromium is
different compared to the support horizon for Node.js. As a result, Node.js
needs to support multiple versions of V8 longer than what upstream needs
to support. V8 branches in Node.js lack of an official maintenance process due
to a missing LTS supported branch.
This document attempts to outline the current maintenance processes, proposes
a workflow for maintaining the V8 branches in both Node.js LTS and current
releases, and discusses how the Node.js and V8 teams at Google can help.
## V8 release schedule
V8 and Chromium follow a
[roughly 4-week release cadence][ChromiumReleaseCalendar]. At any given time
there are several V8 branches that are **active**, see the
[V8 release process](https://v8.dev/docs/release-process). All older branches
are abandoned and are not maintained by the V8 team.
### V8 merge process overview
The process for backporting bug fixes to active branches is officially
documented [on the V8 wiki][V8MergingPatching]. The summary of the process is:
* V8 only supports active branches. There is no testing done on any branches
older than the current stable/beta/master.
* A fix needing backport is tagged w/ _merge-request-x.x_ tag. This can be done
by anyone interested in getting the fix backported. Issues with this tag are
reviewed by the V8 team regularly as candidates for backporting.
* Fixes need some 'baking time' before they can be approved for backporting.
This means waiting a few days to ensure that no issues are detected on the
canary/beta builds.
* Once ready, the issue is tagged w/ _merge-approved-x.x_ and one can do the
actual merge by using the scripts on the [wiki page][V8MergingPatching].
* Merge requests to an abandoned branch will be rejected.
* Only bug fixes are accepted for backporting.
## Node.js support requirements
At any given time Node.js needs to be maintaining a few different V8 branches
for the various Current, LTS, and nightly releases.
The versions of V8 used in Node.js may have already been
abandoned by upstream V8. However, Node.js needs to continue supporting
these branches for many months (Current branches) or several
years (LTS branches).
## Maintenance process
Once a bug in Node.js has been identified to be caused by V8, the first step is
to identify the versions of Node.js and V8 affected. The bug may be present in
multiple different locations, each of which follows a slightly different
process.
* Unfixed bugs. The bug exists in the V8 master branch.
* Fixed, but needs backport. The bug may need porting to one or more branches.
* Backporting to active branches.
* Backporting to abandoned branches.
* Backports identified by the V8 team. Bugs identified by upstream V8 that we
haven't encountered in Node.js yet.
### Unfixed upstream bugs
If the bug can be reproduced on the [Node.js `canary` branch][], Chromium
canary, or V8 tip-of-tree, and the test case is valid, then the bug needs to be
fixed upstream first.
* Start by opening a bug upstream using [this template][V8TemplateUpstreamBug].
* Make sure to include a link to the corresponding Node.js issue
(if one exists).
* If the fix is simple enough, you may fix it yourself;
[contributions][V8Contributing] are welcome.
* V8's build waterfall tests your change.
* Once the bug is fixed it may still need backporting, if it exists in other V8
branches that are still active or are branches that Node.js cares about.
Follow the process for backporting below.
### Backporting to active branches
If the bug exists in any of the active V8 branches, we may need to get the fix
backported. At any given time, there are [two active branches][V8ActiveBranches]
(beta and stable) in addition to master. The following steps are needed to
backport the fix:
* Identify which version of V8 the bug was fixed in.
* Identify if any active V8 branches still contain the bug:
* A tracking bug is needed to request a backport.
* If there isn't already a V8 bug tracking the fix, open a new merge request
bug using this [Node.js specific template][V8TemplateMergeRequest].
* If a bug already exists
* Add a reference to the GitHub issue.
* Attach _merge-request-x.x_ labels to the bug for any active branches
that still contain the bug.
* Once the merge has been approved, it should be merged using the
[merge script documented in the V8 wiki][V8MergingPatching]. Merging requires
commit access to the V8 repository. If you don't have commit access you can
indicate someone on the V8 team can do the merge for you.
* It is possible that the merge request may not get approved, for example if it
is considered to be a feature or otherwise too risky for V8 stable. In such
cases we float the patch on the Node.js side. See the process on 'Backporting
to Abandoned branches'.
* Once the fix has been merged upstream, it can be picked up during an update of
the V8 branch (see below).
### Backporting to abandoned branches
Abandoned V8 branches are supported in the Node.js repository. The fix needs
to be cherry-picked in the Node.js repository and V8-CI must test the change.
As an example for how to backport changes, consider the bug
[RegExp show inconsistent result with other browsers](https://crbug.com/v8/5199).
From the bug we can see that it was merged by V8 into 5.2 and 5.3, and not into
V8 5.1 (since it was already abandoned). Since Node.js `v6.x` uses V8 5.1, the
fix needed to be backported.
#### Backporting with `git-node` (recommended)
You can use [`git-node`][] to help you backport patches. This removes
some manual steps and is recommended.
Here are the steps for the bug mentioned above:
1. Install `git-node` by installing [`@node-core/utils`][].
2. Install the prerequisites for [`git-node-v8`][].
3. Find the commit hash linked-to in the issue (in this case a51f429).
4. Checkout a branch off the appropriate _vY.x-staging_ branch (e.g.
_v6.x-staging_ to fix an issue in V8 5.1).
5. Run `git node v8 backport a51f429`.
6. If there are conflicts, `git-node` will wait for you to resolve them:
```console
$ git node v8 backport a51f429
✔ Update local V8 clone
❯ V8 commit backport
✔ Get current V8 version
✔ Generate patches
❯ Apply and commit patches to deps/v8
❯ Commit a51f429772d1
⠏ Apply patch
◼ Increment embedder version number
◼ Commit patch
? Resolve merge conflicts and enter 'RESOLVED' ‣
```
Resolve conflicts, stage the files (you may need to open another terminal or use
a GUI git client), then return to the terminal running `git-node`, type
`RESOLVED`, and hit <kbd>Enter</kbd>.
7. After you resolve conflicts (or if there are no conflicts), the
output should look like this:
```console
$ git node v8 backport a51f429
✔ Update local V8 clone
✔ V8 commit backport
```
8. Open a PR against the v6.x-staging branch in the Node.js repository.
Launch the normal and [V8 CI][] using the Node.js CI system. We only
needed to backport to v6.x as the other LTS branches weren't affected
by this bug.
See [`git-node-v8-backport`][] for more documentation and additional options.
#### Backporting manually
* For each abandoned V8 branch corresponding to an LTS branch that is affected
by the bug:
* Checkout a branch off the appropriate _vY.x-staging_ branch (e.g.
_v6.x-staging_ to fix an issue in V8 5.1).
* Cherry-pick the commit(s) from the V8 repository.
* Increase the `v8_embedder_string` number in `common.gypi`.
* In some cases the patch may require extra effort to merge in case V8 has
changed substantially. For important issues, we may be able to lean on the
V8 team to get help with reimplementing the patch.
* Open a cherry-pick pull request on `nodejs/node` targeting the
_vY.x-staging_ branch and notify the `@nodejs/v8` team.
* Run the Node.js [V8 CI][] in addition to the [Node.js CI][].
The CI uses the `test-v8` target in the `Makefile`, which uses
`tools/make-v8.sh` to reconstruct a git tree in the `deps/v8` directory to
run V8 tests.[^1]
Here are the steps for the bug mentioned above:
* Download and apply the commit linked-to in the issue (in this case a51f429):
```bash
curl -L https://github.com/v8/v8/commit/a51f429.patch | git am -3 --directory=deps/v8
```
If the branches have diverged significantly, this may not apply cleanly. It
may help to try to cherry-pick the merge to the oldest branch that was done
upstream in V8. In this example, this would be the patch from the merge to
5.2. The hope is that this would be closer to the V8 5.1, and has a better
chance of applying cleanly.
* Modify the commit message to match the format we use for V8 backports and
replace yourself as the author. `git commit --amend --reset-author`. You may
want to add extra description if necessary to indicate the impact of the fix
on Node.js. In this case the original issue was descriptive enough. Example:
```console
deps: cherry-pick a51f429 from V8 upstream
Original commit message:
[regexp] Fix case-insensitive matching for one-byte subjects.
The bug occurs because we do not canonicalize character class ranges
before adding case equivalents. While adding case equivalents, we abort
early for one-byte subject strings, assuming that the ranges are sorted.
Which they are not.
R=marja@chromium.org
BUG=v8:5199
Review-Url: https://codereview.chromium.org/2159683002
Cr-Commit-Position: refs/heads/master@{#37833}
Refs: https://github.com/v8/v8/commit/a51f429772d1e796744244128c9feeab4c26a854
PR-URL: https://github.com/nodejs/node/pull/7833
```
* Increase the `v8_embedder_string` number in `common.gypi`.
* Open a PR against the `v6.x-staging` branch in the Node.js repository. Launch
the normal and [V8 CI][] using the Node.js CI system. We only needed to
backport to `v6.x` as the other LTS branches weren't affected by this bug.
### Backports identified by the V8 team
For bugs found through the browser or other channels, the V8 team marks bugs
that might be applicable to the abandoned branches in use by Node.js. This is
done through manual tagging by the V8 team and through an automated process that
tags any fix that gets backported to the stable branch (as it is likely
candidate for backporting further).
Such fixes are tagged with the following labels in the V8 issue tracker:
* `NodeJS-Backport-Review` ([V8][NodeJS-Backport-Review-V8],
[Chromium][NodeJS-Backport-Review-Chromium]): to be reviewed if this is
applicable to abandoned branches in use by Node.js. This list if regularly
reviewed by the Node.js team at Google to determine applicability to Node.js.
* `NodeJS-Backport-Approved` ([V8][NodeJS-Backport-Approved-V8],
[Chromium][NodeJS-Backport-Approved-Chromium]): marks bugs that are deemed
relevant to Node.js and should be backported.
* `NodeJS-Backport-Done` ([V8][NodeJS-Backport-Done-V8],
[Chromium][NodeJS-Backport-Done-Chromium]): Backport for Node.js has been
performed already.
* `NodeJS-Backport-Rejected` ([V8][NodeJS-Backport-Rejected-V8],
[Chromium][NodeJS-Backport-Rejected-Chromium]): Backport for Node.js is not
desired.
The backlog of issues with such is regularly reviewed by the node-team at Google
to shepherd through the backport process. External contributors are welcome to
collaborate on the backport process as well. Some of the bugs may be
security issues and will not be visible to external collaborators.
## Updating V8
Node.js keeps a vendored copy of V8 inside of the deps/ directory. In addition,
Node.js may need to float patches that do not exist upstream. This means that
some care may need to be taken to update the vendored copy of V8.
V8 builds against the version of ICU supplied by Node.js,
see [maintaining-icu.md](./maintaining-icu.md) for special considerations.
Specifically, a V8 update may necessitate an ICU update.
### Minor updates (patch level)
Because there may be floating patches on the version of V8 in Node.js, it is
safest to apply the patch level updates as a patch. For example, imagine that
upstream V8 is at 5.0.71.47 and Node.js is at 5.0.71.32. It would be best to
compute the diff between these tags on the V8 repository, and then apply that
patch on the copy of V8 in Node.js. This should preserve the patches/backports
that Node.js may be floating (or else cause a merge conflict).
#### Applying minor updates with `git-node` (recommended)
1. Install [`git-node`][] by installing [`@node-core/utils`][].
2. Install the prerequisites for [`git-node-v8`][].
3. Run `git node v8 minor` to apply a minor update.
See [`git-node-v8-minor`][] for more documentation and additional options.
#### Applying minor updates manually
The rough outline of the process is:
```bash
# Assuming your fork of Node.js is checked out in $NODE_DIR
# and you want to update the Node.js main branch.
# Find the current (OLD) version in
# $NODE_DIR/deps/v8/include/v8-version.h
cd $NODE_DIR
git checkout main
git merge --ff-only origin/main
git checkout -b V8_NEW_VERSION
curl -L https://github.com/v8/v8/compare/${V8_OLD_VERSION}...${V8_NEW_VERSION}.patch | git apply --directory=deps/v8
# You may want to amend the commit message to describe the nature of the update
```
V8 also keeps tags of the form _5.4-lkgr_ which point to the _Last Known Good
Revision_ from the 5.4 branch that can be useful in the update process above.
### Major updates
We upgrade the version of V8 in Node.js `main` whenever a V8 release goes stable
upstream, that is, whenever a new release of Chrome comes out.
Upgrading major versions would be much harder to do with the patch mechanism
above. A better strategy is to
1. Audit the current `main` branch and look at the patches that have been
floated since the last major V8 update.
2. Replace the copy of V8 in Node.js with a fresh checkout of the latest stable
V8 branch. Special care must be taken to recursively update the DEPS that V8
has a compile time dependency on (at the moment of this writing, these are
only trace\_event and gtest\_prod.h)
3. Reset the `v8_embedder_string` variable to "-node.0" in `common.gypi`.
4. Refloat (cherry-pick) all the patches from list computed in 1) as necessary.
Some of the patches may no longer be necessary.
To audit for floating patches:
```bash
git log --oneline deps/v8
```
To replace the copy of V8 in Node.js, use the [`git-node`][] tool. For example,
if you want to replace the copy of V8 in Node.js with the branch-head for V8 5.1
branch:
```bash
cd $NODE_DIR
git node v8 major --branch=5.1-lkgr
```
This should be followed up with manual refloating of all relevant patches.
## Proposal: Using a fork repository to track upstream V8
The fact that Node.js keeps a vendored, potentially edited copy of V8 in deps/
makes the above processes a bit complicated. An alternative proposal would be to
create a fork of V8 at `nodejs/v8` that would be used to maintain the V8
branches. This has several benefits:
* The process to update the version of V8 in Node.js could be automated to track
the tips of various V8 branches in `nodejs/v8`.
* It would simplify cherry-picking and porting of fixes between branches as the
version bumps in `v8-version.h` would happen as part of this update instead of
on every change.
* It would simplify the V8-CI and make it more automatable.
* The history of the V8 branch in `nodejs/v8` becomes purer and it would make it
easier to pull in the V8 team for help with reviewing.
* It would make it simpler to setup an automated build that tracks Node.js
`main` + V8 lkgr integration build.
This would require some tooling to:
* A script that would update the V8 in a specific Node.js branch with V8 from
upstream (dependent on branch abandoned vs. active).
* We need a script to bump V8 version numbers when a new version of V8 is
promoted from `nodejs/v8` to `nodejs/node`.
* Enabled the V8-CI build in Jenkins to build from the `nodejs/v8` fork.
[^1]: On macOS, the V8 tests require a full Xcode install, not just the "command
line tools" for Xcode.
[ChromiumReleaseCalendar]: https://www.chromium.org/developers/calendar
[Node.js CI]: https://ci.nodejs.org/job/node-test-pull-request/
[Node.js `canary` branch]: https://github.com/nodejs/node-v8/tree/canary
[NodeJS-Backport-Approved-Chromium]: https://bugs.chromium.org/p/chromium/issues/list?can=1&q=label%3ANodeJS-Backport-Approved
[NodeJS-Backport-Approved-V8]: https://bugs.chromium.org/p/v8/issues/list?can=1&q=label%3ANodeJS-Backport-Approved
[NodeJS-Backport-Done-Chromium]: https://bugs.chromium.org/p/chromium/issues/list?can=1&q=label%3ANodeJS-Backport-Done
[NodeJS-Backport-Done-V8]: https://bugs.chromium.org/p/v8/issues/list?can=1&q=label%3ANodeJS-Backport-Done
[NodeJS-Backport-Rejected-Chromium]: https://bugs.chromium.org/p/chromium/issues/list?can=1&q=label%3ANodeJS-Backport-Rejected
[NodeJS-Backport-Rejected-V8]: https://bugs.chromium.org/p/v8/issues/list?can=1&q=label%3ANodeJS-Backport-Rejected
[NodeJS-Backport-Review-Chromium]: https://bugs.chromium.org/p/chromium/issues/list?can=1&q=label%3ANodeJS-Backport-Review
[NodeJS-Backport-Review-V8]: https://bugs.chromium.org/p/v8/issues/list?can=1&q=label%3ANodeJS-Backport-Review
[V8 CI]: https://ci.nodejs.org/job/node-test-commit-v8-linux/
[V8ActiveBranches]: https://build.chromium.org/p/client.v8.branches/console
[V8Contributing]: https://v8.dev/docs/contribute
[V8MergingPatching]: https://v8.dev/docs/merge-patch
[V8TemplateMergeRequest]: https://bugs.chromium.org/p/v8/issues/entry?template=Node.js%20merge%20request
[V8TemplateUpstreamBug]: https://bugs.chromium.org/p/v8/issues/entry?template=Node.js%20upstream%20bug
[`@node-core/utils`]: https://github.com/nodejs/node-core-utils#Install
[`git-node-v8-backport`]: https://github.com/nodejs/node-core-utils/blob/main/docs/git-node.md#git-node-v8-backport-sha
[`git-node-v8-minor`]: https://github.com/nodejs/node-core-utils/blob/main/docs/git-node.md#git-node-v8-minor
[`git-node-v8`]: https://github.com/nodejs/node-core-utils/blob/HEAD/docs/git-node.md#git-node-v8
[`git-node`]: https://github.com/nodejs/node-core-utils/blob/HEAD/docs/git-node.md#git-node-v8

View File

@ -0,0 +1,59 @@
# Maintaining cjs-module-lexer
The [cjs-module-lexer](https://github.com/nodejs/node/tree/HEAD/deps/cjs-module-lexer)
dependency is used within the Node.js ESM implementation to detect the
named exports of a CommonJS module.
It is used within
[`node:internal/modules/esm/translators`](https://github.com/nodejs/node/blob/HEAD/lib/internal/modules/esm/translators.js)
in which both `internal/deps/cjs-module-lexer/lexer` and
`internal/deps/cjs-module-lexer/dist/lexer` are required and used.
`internal/deps/cjs-module-lexer/lexer`
is a regular JavaScript implementation that is
used when WebAssembly is not available on a platform.
`internal/deps/cjs-module-lexer/dist/lexer` is a faster
implementation using WebAssembly which is generated from a
C based implementation. These two paths
resolve to the files in `deps/cjs-module-lexer` due to their
inclusion in the `deps_files` entry in
[node.gyp](https://github.com/nodejs/node/blob/main/node.gyp).
The two different versions of lexer.js are maintained in the
[nodejs/cjs-module-lexer][] project.
In order to update the Node.js dependencies to use to a newer version
of cjs-module-lexer, complete the following steps:
* Clone [nodejs/cjs-module-lexer][]
and check out the version that you want Node.js to use.
* Follow the WASM build steps outlined in
[wasm-build-steps](https://github.com/nodejs/cjs-module-lexer#wasm-build-steps).
This will generate the WASM based dist/lexer.js file.
* Preserving the same directory structure, copy the following files over
to `deps/cjs-module-lexer` directory where you have checked out Node.js.
```text
├── CHANGELOG.md
├── dist
│ ├── lexer.js
│ └── lexer.mjs
├── lexer.js
├── LICENSE
├── package.json
└── README.md
```
* Update the link to the cjs-module-lexer in the list at the end of
[doc/api/esm.md](../../api/esm.md)
to point to the updated version.
* Create a PR, adding the files in the deps/cjs-module-lexer that
were modified.
If updates are needed to cjs-module-lexer for Node.js, first PR
those updates into
[nodejs/cjs-module-lexer][],
request a release and then pull in the updated version once available.
[nodejs/cjs-module-lexer]: https://github.com/nodejs/cjs-module-lexer

View File

@ -0,0 +1,438 @@
# Maintaining Dependencies
Node.js depends on additional components beyond the Node.js code
itself. These dependencies provide both native and JavaScript code
and are built together with the code under the `src` and `lib`
directories to create the Node.js binaries.
All dependencies are located within the `deps` directory.
This a list of all the dependencies:
* [acorn][]
* [ada][]
* [amaro][]
* [base64][]
* [brotli][]
* [c-ares][]
* [cjs-module-lexer][]
* [corepack][]
* [googletest][]
* [histogram][]
* [icu-small][]
* [inspector\_protocol][inspector_protocol]
* [libuv][]
* [llhttp][]
* [minimatch][]
* [nghttp2][]
* [nghttp3][]
* [ngtcp2][]
* [npm][]
* [openssl][]
* [postject][]
* [simdjson][]
* [sqlite][]
* [undici][]
* [uvwasi][]
* [V8][]
* [zlib][]
* [zstd][]
Any code which meets one or more of these conditions should
be managed as a dependency:
* originates in an upstream project and is maintained
in that upstream project.
* is not built from the `preferred form of the work for
making modifications to it` (see
[GNU GPL v2, section 3.](https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html)
when `make node` is run. A good example is
WASM code generated from C (the preferred form).
Typically generation is only supported on a subset of platforms, needs
additional tools, and is pre-built outside of the `make node`
step and then committed as a WASM binary in the directory
for the dependency under the `deps` directory.
By default all dependencies are bundled into the Node.js
binary, however, `configure` options should be available to
use an externalized version at runtime when:
* the dependency provides native code and is available as
a shared library in one or more of the common Node.js
distributions.
* the dependency provides JavaScript and is not built
from the `preferred form of the work for making modifications
to it` when `make node` is run.
Many distributions use externalized dependencies for one or
more of these reasons:
1. They have a requirement to build everything that they ship
from the `preferred form of the work for making
modifications to it`. This means that they need to
replace any pre-built components (for example WASM
binaries) with an equivalent that they have built.
2. They manage the dependency separately as it is used by
more applications than just Node.js. Linking against
a shared library allows them to manage updates and
CVE fixes against the library instead of having to
patch all of the individual applications.
3. They have a system wide configuration for the
dependency that all applications should respect.
## Supporting externalized dependencies with native code
Support for externalized dependencies with native code for which a
shared library is available can added by:
* adding options to `configure.py`. These are added to the
shared\_optgroup and include an options to:
* enable use of a shared library
* set the name of the shared library
* set the path to the directory with the includes for the
shared library
* set the path to where to find the shared library at
runtime
* add a call to configure\_library() to `configure.py` for the
library at the end of list of existing configure\_library() calls.
If there are additional libraries that are required it is
possible to list more than one with the `pkgname` option.
* in `node.gypi` guard the build for the dependency
with `node_shared_depname` so that it is only built if
the dependency is being bundled into Node.js itself. For example:
```text
[ 'node_shared_brotli=="false"', {
'dependencies': [ 'deps/brotli/brotli.gyp:brotli' ],
}],
```
## Supporting externalizable dependencies with JavaScript code
Support for an externalizable dependency with JavaScript code
can be added by:
* adding an entry to the `shareable_builtins` map in
`configure.py`. The path should correspond to the file
within the deps directory that is normally bundled into
Node.js. For example `deps/cjs-module-lexer/lexer.js`.
This will add a new option for building with that dependency
externalized. After adding the entry you can see
the new option by running `./configure --help`.
* adding a call to `AddExternalizedBuiltin` to the constructor
for BuiltinLoader in `src/node_builtins.cc` for the
dependency using the `NODE_SHARED_BUILTLIN` #define generated for
the dependency. After running `./configure` with the new
option you can find the #define in `config.gypi`. You can cut and
paste one of the existing entries and then update to match the
import name for the dependency and the #define generated.
* if the version of the dependency is reported in `process.versions`,
update `src/node_metadata.h` and `src/node_metadata.cc` so that the
version is not reported when the dependency is externalized.
Not reporting the version is better than incorrectly reporting
the version of the dependency bundled with Node.js, instead of the
version for the externalized dependency. Use one of the existing
externalized dependencies, like Undici, as an example of how to
update these files correctly. Make sure to run the tests with the
dependency externalized, as the tests will also need to be updated
to handle this properly.
## Supporting non-externalized dependencies with JavaScript code
If the dependency consists of JavaScript in the
`preferred form of the work for making modifications to it`, it
can be added as a non-externalizable dependency. In this case
simply add the path to the JavaScript file in the `deps_files`
list in the `node.gyp` file.
## Common approach for dependencies with WASM components
WASM components within dependencies are most often built
outside of the regular Node.js `make build` step. They also
require different tools.
It is important that the tools and their versions used to build
WASM components shipped within Node.js are well documented and
be available if needed to rebuild/update older Node.js versions.
In order to minimize the different number of tools and versions
used to build WASM components and to document and ensure future
availability, the project builds and maintains a common
[wasm-builder](https://github.com/nodejs/wasm-builder) container
that should be use to build WASM components in Node.js
dependencies.
The container provides a durable copy of the versions of the tools
used for a specific build which are under the control of the Node.js
project. In addition, the tools and verions are documented through metadata
within the container in the `/home/node/metadata directory`.
The available tools can be found by looking at the current version of the
[Dockerfile](https://github.com/nodejs/wasm-builder/blob/main/container-build-info/Dockerfile)
used to create the container.
If additional WASM tool are needed beyond those available in the
container, additions should be PR'd into the wasm-builder container.
Examples of using the container include:
* [build/wasm.js](https://github.com/nodejs/undici/blob/main/build/wasm.js) from undici
* [tools/build-wasm.js](https://github.com/nodejs/amaro/blob/main/tools/build-wasm.js) from amaro
In addition to using the container to build WASM components, the goal is also
for the WASM components and final files that are shipped with Node.js to be
built by the [dep-updaters](https://github.com/nodejs/node/tree/main/tools/dep_updaters)
that are run on a regular basis and that they use only the files available in the Node.js
repo for the dependency. For example, being able to rebuild the WASM and files that
we ship in Node.js using only the files in
[../deps/undici](https://github.com/nodejs/node/tree/main/deps/undici).
## Updating dependencies
Most dependencies are automatically updated by
[dependency-update-action][] that runs weekly.
However, it is possible to manually update a dependency by running
the corresponding script in `tools/update-deps`.
[OpenSSL](https://github.com/openssl/openssl) has its own update action:
[update-openssl-action][].
[npm-cli-bot](https://github.com/npm/cli/blob/latest/.github/workflows/create-node-pr.yml)
takes care of npm update, it is maintained by the npm team.
PRs for manual dependency updates should only be accepted if
the update cannot be generated by the automated tooling,
the reason is clearly documented and either the PR is
reviewed in detail or it is from an existing collaborator.
In general updates to dependencies should only be accepted
if they have already landed in the upstream. The TSC may
grant an exception on a case-by-case basis. This avoids
the project having to float patches for a long time and
ensures that tooling can generate updates automatically.
## Dependency list
### acorn
The [acorn](https://github.com/acornjs/acorn) dependency is a JavaScript parser.
[acorn-walk](https://github.com/acornjs/acorn/tree/master/acorn-walk) is
an abstract syntax tree walker for the ESTree format.
### ada
The [ada](https://github.com/ada-url/ada) dependency is a
fast and spec-compliant URL parser written in C++.
### amaro
The [amaro](https://www.npmjs.com/package/amaro) dependency is a wrapper around the
WebAssembly version of the SWC JavaScript/TypeScript parser.
### brotli
The [brotli](https://github.com/google/brotli) dependency is
used for the homonym generic-purpose lossless compression algorithm.
### c-ares
The [c-ares](https://github.com/c-ares/c-ares) is a C library
for asynchronous DNS requests.
### cjs-module-lexer
The [cjs-module-lexer](https://github.com/nodejs/node/tree/HEAD/deps/cjs-module-lexer)
dependency is used within the Node.js ESM implementation to detect the
named exports of a CommonJS module.
See [maintaining-cjs-module-lexer][] for more information.
### corepack
The [corepack](https://github.com/nodejs/corepack) dependency is a
zero-runtime-dependency Node.js script that acts as a bridge between
Node.js projects and the package managers they are intended to
be used with during development.
In practical terms, Corepack will let you use Yarn and pnpm without having to
install them - just like what currently happens with npm, which is shipped
by Node.js by default.
### googletest
The [googletest](https://github.com/google/googletest) dependency is Google’s
C++ testing and mocking framework.
### histogram
The [histogram](https://github.com/HdrHistogram/HdrHistogram_c) dependency is
a C port of High Dynamic Range (HDR) Histogram.
### ic
The [icu](http://site.icu-project.org) is widely used set of C/C++
and Java libraries providing Unicode and Globalization
support for software applications.
See [maintaining-icu][] for more information.
### inspector\_protocol
The [inspector\_protocol](https://chromium.googlesource.com/deps/inspector_protocol/)
is Chromium's of code generators and templates for the inspector protocol.
See [this doc](../../../tools/inspector_protocol/README.md) for more information.
### libuv
The [libuv](https://github.com/libuv/libuv) dependency is a
multi-platform support library with a focus on asynchronous I/O.
It was primarily developed for use by Node.js.
### llhttp
The [llhttp](https://github.com/nodejs/llhttp) dependency is
the http parser used by Node.js.
See [maintaining-http][] for more information.
### minimatch
The [minimatch](https://github.com/isaacs/minimatch) dependency is a
minimal matching utility.
### nghttp2
The [nghttp2](https://github.com/nghttp2/nghttp2) dependency is a C library
implementing HTTP/2 protocol.
See [maintaining-http][] for more information.
### nghttp3
The [nghttp3](https://github.com/ngtcp2/nghttp3) dependency is HTTP/3 library
written in C. See ngtcp2 for more information.
### ngtcp2
The ngtcp2 and nghttp3 dependencies provide the core functionality for
QUIC and HTTP/3.
The sources are pulled from:
* ngtcp2: <https://github.com/ngtcp2/ngtcp2>
* nghttp3: <https://github.com/ngtcp2/nghttp3>
In both the `ngtcp2` and `nghttp3` git repos, the active development occurs
in the default branch (currently named `main` in each). Tagged versions do not
always point to the default branch.
We only use a subset of the sources for each.
The `nghttp3` library depends on `ngtcp2`. Both should always be updated
together. From `ngtcp2` we only want the contents of the `lib` and `crypto`
directories; from `nghttp3` we only want the contents of the `lib` directory.
### npm
The [npm](https://github.com/npm/cli) dependency is
the package manager for JavaScript.
New pull requests should be opened when a "next" version of npm has
been released. Once the "next" version has been promoted to "latest"
the PR should be updated as necessary.
The specific Node.js release streams the new version will be able to land into
are at the discretion of the release and LTS teams.
This process only covers full updates to new versions of npm. Cherry-picked
changes can be reviewed and landed via the normal consensus seeking process.
### openssl
The [openssl](https://github.com/quictls/openssl) dependency is a
fork of OpenSSL to enable QUIC.
[OpenSSL](https://www.openssl.org/) is toolkit for general-purpose
cryptography and secure communication.
Node.js currently uses the quictls/openssl fork, which closely tracks
the main openssl/openssl releases with the addition of APIs to support
the QUIC protocol.
See [maintaining-openssl][] for more information.
### postject
The [postject](https://github.com/nodejs/postject) dependency is used for the
[Single Executable strategic initiative](https://github.com/nodejs/single-executable).
### simdjson
The [simdjson](https://github.com/simdjson/simdjson) dependency is
a C++ library for fast JSON parsing.
### sqlite
The [sqlite](https://github.com/sqlite/sqlite) dependency is
an embedded SQL database engine written in C.
### undici
The [undici](https://github.com/nodejs/undici) dependency is an HTTP/1.1 client,
written from scratch for Node.js..
See [maintaining-http][] for more information.
### uvwasi
The [uvwasi](https://github.com/nodejs/uvwasi) dependency implements
the WASI system call API, so that WebAssembly runtimes can easily
implement WASI calls.
Under the hood, uvwasi leverages libuv where possible for maximum portability.
See [maintaining-web-assembly][] for more information.
### V8
[V8](https://chromium.googlesource.com/v8/v8.git/) is Google's open source
high-performance JavaScript and WebAssembly engine, written in C++.
See [maintaining-V8][] for more information.
### zlib
The [zlib](https://chromium.googlesource.com/chromium/src/+/refs/heads/main/third_party/zlib)
dependency lossless data-compression library,
it comes from the Chromium team's zlib fork which incorporated
performance improvements not currently available in standard zlib.
### zstd
The [zstd](https://github.com/facebook/zstd) dependency is used for compression
according to [RFC 8878](https://datatracker.ietf.org/doc/html/rfc8878).
[acorn]: #acorn
[ada]: #ada
[amaro]: #amaro
[base64]: #base64
[brotli]: #brotli
[c-ares]: #c-ares
[cjs-module-lexer]: #cjs-module-lexer
[corepack]: #corepack
[dependency-update-action]: ../../../.github/workflows/tools.yml
[googletest]: #googletest
[histogram]: #histogram
[icu-small]: #icu-small
[inspector_protocol]: #inspector_protocol
[libuv]: #libuv
[llhttp]: #llhttp
[maintaining-V8]: ./maintaining-V8.md
[maintaining-cjs-module-lexer]: ./maintaining-cjs-module-lexer.md
[maintaining-http]: ./maintaining-http.md
[maintaining-icu]: ./maintaining-icu.md
[maintaining-openssl]: ./maintaining-openssl.md
[maintaining-web-assembly]: ./maintaining-web-assembly.md
[minimatch]: #minimatch
[nghttp2]: #nghttp2
[nghttp3]: #nghttp3
[ngtcp2]: #ngtcp2
[npm]: #npm
[openssl]: #openssl
[postject]: #postject
[simdjson]: #simdjson
[sqlite]: #sqlite
[undici]: #undici
[update-openssl-action]: ../../../.github/workflows/update-openssl.yml
[uvwasi]: #uvwasi
[v8]: #v8
[zlib]: #zlib
[zstd]: #zstd

View File

@ -0,0 +1,90 @@
# Maintaining HTTP
Support for HTTP is a key priority in terms of ensuring the continued success of
Node.js as captured in the project's
[technical priorities](https://github.com/nodejs/node/blob/HEAD/doc/contributing/technical-priorities.md).
The current high level strategy is based on the discussion in the
[Next-10](https://github.com/nodejs/next-10)
[mini-summit](https://github.com/nodejs/next-10/blob/main/meetings/summit-jan-2022.md)
on modern HTTP which was held on Jan 27 2022.
## High level strategy
The key elements of our strategy for future HTTP APIs are:
* APIs should be HTTP protocol independent (support HTTP1, HTTP2, etc.).
* APIs should be transport protocol independent (TCP, QUIC, etc.).
* APIs should provide good defaults that perform well.
* Client/server APIs should be consistent and allow easy integration.
* Common requirements like piping out from client API to server APIs should be
easy.
* For both the Client and Server there is a need for multiple APIs, with each
targeting a different level of abstraction.
Unfortunately our existing HTTP APIs (
[HTTP](https://nodejs.org/docs/latest/api/http.html),
[HTTPS](https://nodejs.org/docs/latest/api/https.html), and
[HTTP2](https://nodejs.org/docs/latest/api/http2.html))
do not align with our high level strategy. While these APIs
are widely used and we do not plan to deprecate or remove them,
they are not the focus of active development or performance improvements.
Bug fixes however are still important for all of these APIs.
With respect to the HTTP protocols themselves, our current assessment in
terms of priority within existing or new APIs is:
* HTTP 2 is in “maintenance mode” for both protocol and APIs.
* HTTP 1 is a stable protocol, but innovation is still possible with the APIs.
* HTTP 3 is an important protocol and we need to add support for it.
The current strategy is to build out 2 new client APIs and 2 new Server APIs
in line with the high level strategy above.
While transport-level APIs are important (e.g. socket level), the HTTP APIs
should not be tied to a specific transport-level API. Therefore,
transport-level APIs are out of the scope of our HTTP strategy/maintenance
information.
### Client APIs
For client APIs we want a high-level API and a low-level API when
more control is required. The current plan is for the following APIs:
* High-level API -
[Fetch](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API)
based API built on top of [undici](https://www.npmjs.com/package/undici).
* Low-level API - a subset of the APIs exposed by
[undici](https://www.npmjs.com/package/undici). The exact shape and
set of these APIs is still to be worked out. The current plan is to
pull undici into Node.js core without exposing its APIs in the Node.js
API so that it can initially be used to support the higher level
Fetch-based API. As this gets worked out we will discuss which
APIs to expose in the Node.js API surface.
### Server APIs
For the server APIs we do not yet have a clear path, other than wanting
to align them with the APIs built for the client.
### HTTP, HTTPS
The low-level implementation of the
[HTTP](https://nodejs.org/docs/latest/api/http.html)
and [HTTPS](https://nodejs.org/docs/latest/api/https.html) APIs
are maintained in the [llhttp](https://github.com/nodejs/llhttp)
repository. Updates are pulled into Node.js under
[deps/llhttp](https://github.com/nodejs/node/tree/HEAD/deps/llhttp).
### HTTP2
The low-level implementation of
[HTTP2](https://nodejs.org/docs/latest/api/http2.html)
is based on [nghttp2](https://nghttp2.org/). Updates are pulled into Node.js
under [deps/nghttp2](https://github.com/nodejs/node/tree/HEAD/deps/nghttp2)
as needed.
The low-level implementation is made available in the Node.js API through
JavaScript code in the [lib](https://github.com/nodejs/node/tree/HEAD/lib)
directory and C++ code in the
[src](https://github.com/nodejs/node/tree/HEAD/src) directory

View File

@ -0,0 +1,270 @@
# Maintaining ICU in Node.js
## Background
International Components for Unicode ([ICU4C][ICU]) is used both by V8
and also by Node.js directly to provide internationalization
functionality. To quote from icu-project.org:
> ICU is a mature, widely used set of C/C++ and Java libraries providing
> Unicode and Globalization support for software applications. ICU is
> widely portable and gives applications the same results on all platforms
> and between C/C++ and Java software.
If Node.js is configured to use its built-in ICU,
it uses a strict subset of ICU which is in
[deps/icu-small](https://github.com/nodejs/node/tree/HEAD/deps/icu-small).
A good description of the different ways Node.js can be built with ICU
support is in [api/intl.html](https://nodejs.org/api/intl.html).
## Data dependencies
ICU consumes and includes:
* Extracted locale data from [CLDR][]
* Extracted [Unicode][] data.
* Time zone ([tz][]) data
The current versions of these items can be viewed for node with `node -p process.versions`:
```console
$ node -p process.versions
{
…
cldr: '35.1',
icu: '64.2',
tz: '2019a',
unicode: '12.1'
}
```
### Time zone data
Time zone data files are updated independently of ICU CLDR data. ICU and its
main data files do not need to be upgraded in order to apply time zone data file
fixes.
The [IANA tzdata][tz] project releases new versions and announces them on the
[`tz-announce`](https://mm.icann.org/pipermail/tz-announce/) mailing list.
The Unicode project takes new releases and publishes
[updated time zone data files](https://github.com/unicode-org/icu-data/tree/HEAD/tzdata/icunew)
in the icu/icu-data repository.
All modern versions of Node.js use the version 44 ABI of the time zone data
files.
#### Example: updating the ICU `.dat` file
* Decompress `deps/icu-small/source/data/in/icudt##l.dat.bz2`, where `##` is
the ICU major version number.
* Clone the icu/icu-data repository and copy the latest `tzdata` release `le`
files into the `source/data/in` directory.
* Follow the upstream [ICU instructions](https://unicode-org.github.io/icu/userguide/datetime/timezone/)
to patch the ICU `.dat` file:
> `for i in zoneinfo64.res windowsZones.res timezoneTypes.res metaZones.res;
> do icupkg -a $i icudt*l.dat`
* Optionally, verify that there is only one of the above files listed when using
`icupkg -l`.
* Optionally, extract each file using `icupkg -x` and verify the `shasum`
matches the desired value.
* Compress the `.dat` file with the same filename as in the first step.
* Build, test, verifying `process.versions.tz` matches the desired version.
* Create a new minor version release.
## Release schedule
ICU typically has >1 release a year, particularly coinciding with a major
release of [Unicode][]. The current release schedule is available on the [ICU][]
website on the left sidebar.
### V8 depends on ICU
V8 will aggressively upgrade to a new ICU version, due to requirements for
features/bugfixes needed for [Ecma402][] support. The minimum required version
of ICU is specified within the V8 source tree. If the ICU version is too old,
V8 will not compile.
```c
// deps/v8/src/objects/intl-objects.h
#define V8_MINIMUM_ICU_VERSION 65
```
V8 in Node.js depends on the ICU version supplied by Node.js.
The file `tools/icu/icu_versions.json` contains the current minimum
version of ICU that Node.js is known to work with. This should be
_at least_ the same version as V8, so that users will find out
earlier that their ICU is too old. A test case validates this when
Node.js is built.
## How to upgrade ICU
> The script `tools/dep_updaters/update-icu.sh` automates
> this process.
* Make sure your Node.js workspace is clean (`git status`
should be sufficient).
* Configure Node.js with the specific [ICU version](http://site.icu-project.org/download)
you want to upgrade to, for example:
```bash
./configure \
--with-intl=full-icu \
--with-icu-source=https://github.com/unicode-org/icu/releases/download/release-67-1/icu4c-67_1-src.tgz
make
```
> _Note_ in theory, the equivalent `vcbuild.bat` commands should work also,
> but the commands below are makefile-centric.
* If there are ICU version-specific changes needed, you may need to make changes
in `tools/icu/icu-generic.gyp` or add patch files to `tools/icu/patches`.
* Specifically, look for the lists in `sources!` in the `tools/icu/icu-generic.gyp` for
files to exclude.
* Verify the Node.js build works:
```bash
make test-ci
```
Also running
<!-- eslint-disable strict -->
```js
new Intl.DateTimeFormat('es', { month: 'long' }).format(new Date(9E8));
```
…Should return `enero` not `January`.
* Now, run the shrink tool to update `deps/icu-small` from `deps/icu`
> :warning: Do not modify any source code in `deps/icu-small` !
> See section below about floating patches to ICU.
```bash
python tools/icu/shrink-icu-src.py
```
* Now, do a clean rebuild of Node.js to test:
```bash
make -k distclean
./configure
make
```
* Test this newly default-generated Node.js
<!-- eslint-disable strict -->
```js
process.versions.icu;
new Intl.DateTimeFormat('es', { month: 'long' }).format(new Date(9E8));
```
(This should print your updated ICU version number, and also `enero` again.)
You are ready to check in (`git add`) the updated `deps/icu-small`.
> :warning: Do not modify any source code in `deps/icu-small` !
> See section below about floating patches to ICU.
* Now, rebuild the Node.js license.
```bash
# clean up - remove deps/icu
make clean
tools/license-builder.sh
```
* Update the URL and hash for the full ICU file in `tools/icu/current_ver.dep`.
It should match the ICU URL used in the first step. When this is done, the
following should build with small ICU.
```bash
# clean up
rm -rf out deps/icu deps/icu4c*
./configure --with-intl=small-icu --download=all
make
make test-ci
```
* Commit the change to the `deps/icu-small`, `tools/icu/current_ver.dep`
and `LICENSE` files.
## Floating patches to ICU
Floating patches are applied at `configure` time. The "patch" files
are used instead of the original source files. The patch files are
complete `.cpp` files replacing the original contents.
Patches are tied to a specific ICU version. They won't apply to a
future ICU version. We assume that you filed a bug against [ICU][] and
upstreamed the fix, so the patch won't be needed in a later ICU
version.
### Example
For example, to patch `source/tools/toolutil/pkg_genc.cpp` for
ICU version 63:
```bash
# go to your Node.js source directory
cd <node>
# create the floating patch directory
mkdir -p tools/icu/patches/63
# create the subdirectory for the file(s) to patch:
mkdir -p tools/icu/patches/63/source/tools/toolutil/
# copy the file to patch
cp deps/icu-small/source/tools/toolutil/pkg_genc.cpp \
tools/icu/patches/63/source/tools/toolutil/pkg_genc.cpp
# Make any changes to this file:
(edit tools/icu/patches/63/source/tools/toolutil/pkg_genc.cpp )
# test
make clean && ./configure && make
```
You should see a message such as:
```console
INFO: Using floating patch "tools/icu/patches/63/source/tools/toolutil/pkg_genc.cpp" from "tools/icu"
```
### Clean up
Any patches older than the minimum version given in `tools/icu/icu_versions.json`
ought to be deleted, because they will never be used.
### Why not just modify the ICU source directly?
Especially given the V8 dependencies above, there may be times when a floating
patch to ICU is required. Though it seems expedient to simply change a file in
`deps/icu-small`, this is not the right approach for the following reasons:
1. **Repeatability.** Given the complexity of merging in a new ICU version,
following the steps above in the prior section of this document ought to be
repeatable without concern for overriding a patch.
2. **Verifiability.** Given the number of files modified in an ICU PR,
a floating patch could easily be missed or dropped altogether next time
something is landed.
3. **Compatibility.** There are a number of ways that ICU can be loaded into
Node.js (see the top level README.md). Only modifying `icu-small` would cause
the patch not to be landed in case the user specifies the ICU source code
another way.
[CLDR]: http://cldr.unicode.org/
[Ecma402]: https://github.com/tc39/ecma402
[ICU]: http://site.icu-project.org/
[Unicode]: https://home.unicode.org/
[tz]: https://www.iana.org/time-zones

View File

@ -0,0 +1,133 @@
# Maintaining OpenSSL
OpenSSL is automatically updated by the [update-openssl-action][].
There is also a script in `tools/dep_updaters` that can be used to update it.
This document describes how to manually update `deps/openssl/`.
## Requirements
* Linux environment.
* `perl` Only Perl version 5 is tested.
* `nasm` (<https://www.nasm.us/>) Version 2.11 or higher is needed.
* GNU `as` in binutils. Version 2.26 or higher is needed.
## 0. Check requirements
```console
% perl -v
This is perl 5, version 22, subversion 1 (v5.22.1) built for
x86_64-linux-gnu-thread-multi
(with 60 registered patches, see perl -V for more detail)
% as --version
GNU assembler (GNU Binutils for Ubuntu) 2.26.1
Copyright (C) 2015 Free Software Foundation, Inc.
...
% nasm -v
NASM version 2.11.08
```
## 1. Obtain and extract new OpenSSL sources
Get a new source from <https://github.com/openssl/openssl/tree/openssl-3.0.16>
and copy all files into `deps/openssl/openssl`. Then add all files and commit
them. (The link above, and the branch, will change with each new OpenSSL
release).
```bash
git clone https://github.com/openssl/openssl
cd openssl
cd ../node/deps/openssl
rm -rf openssl
cp -R ../../../openssl openssl
rm -rf openssl/.git*
git add --all openssl
git commit openssl
```
```text
deps: upgrade openssl sources to openssl-3.0.16
This updates all sources in deps/openssl/openssl by:
$ git clone git@github.com:openssl/openssl.git
$ cd openssl
$ git checkout openssl-3.0.16
$ cd ../node/deps/openssl
$ rm -rf openssl
$ cp -R ../../../openssl openssl
$ rm -rf openssl/.git*
$ git add --all openssl
$ git commit openssl
```
## 2. Execute `make` in `deps/openssl/config` directory
Use `make` to regenerate all platform dependent files in
`deps/openssl/config/archs/`:
```bash
# On non-Linux machines
make gen-openssl
# On Linux machines
make -C deps/openssl/config clean
make -C deps/openssl/config
```
Fix up 32-bit Windows assembler directives. This will allow the commits to be
cherry-picked to older release lines that still provide binaries on 32-bit Windows.
```bash
make -C deps/openssl/config clean
# Edit deps/openssl/openssl/crypto/perlasm/x86asm.pl changing
# #ifdef to %ifdef to make it compatible to nasm on 32-bit Windows.
# See: https://github.com/nodejs/node/pull/43603#issuecomment-1170670844
# Reference: https://github.com/openssl/openssl/issues/18459
```
## 3. Check diffs
Check diffs to ensure updates are right. Even if there are no updates in openssl
sources, `buildinf.h` files will be updated because they have timestamp
data in them.
```bash
git diff -- deps/openssl
```
_Note_: On Windows, OpenSSL Configure generates a `makefile` that can be
used for the `nmake` command. The `make` command in step 2 (above) uses
`Makefile_VC-WIN64A` and `Makefile_VC-WIN32` that are manually
created. When source files or build options are updated in Windows,
it needs to change these two Makefiles by hand. If you are not sure,
please ask @shigeki for details.
## 4. Commit and make test
Update all architecture dependent files. Do not forget to git add or remove
files if they are changed before committing:
```bash
git add deps/openssl/config/archs
git add deps/openssl/openssl
git commit
```
The commit message can be written as (with the openssl version set
to the relevant value):
```text
deps: update archs files for openssl-3.0.16
After an OpenSSL source update, all the config files need to be
regenerated and committed by:
$ make -C deps/openssl/config
$ git add deps/openssl/config/archs
$ git add deps/openssl/openssl
$ git commit
```
Finally, build Node.js and run the tests.
[update-openssl-action]: ../../../.github/workflows/update-openssl.yml

View File

@ -0,0 +1,140 @@
# Maintaining the root certificates
Node.js contains a compiled-in set of root certificates used as trust anchors
for TLS certificate validation.
The certificates come from Mozilla, specifically NSS's `certdata.txt` file.
The PEM encodings of the certificates are converted to C strings, and committed
in `src/node_root_certs.h`.
## When to update
Root certificates should be updated sometime after Mozilla makes an NSS release,
check the [NSS release schedule][].
## Process
The `tools/dep_updaters/update-root-certs.mjs` script automates the update of
the root certificates, including:
* Downloading `certdata.txt` from Mozilla's source control repository.
* Running `tools/mk-ca-bundle.pl` to convert the certificates and generate
`src/node_root_certs.h`.
* Using `git diff-files` to determine which certificate have been added and/or
removed.
Manual instructions are included in the following collapsed section.
<details>
Commands assume that the current working directory is the root of a checkout of
the nodejs/node repository.
1. Find NSS metadata for update.
The latest released NSS version, release date, Firefox version, and Firefox
release date can be found in the [NSS release schedule][].
The tag to fetch `certdata.txt` from is found by looking for the release
version in the [tag list][].
2. Update `certdata.txt` from the NSS release tag.
Update the tag in the commands below, and run:
```bash
cd tools/
./mk-ca-bundle.pl -v 2>_before
curl -O https://hg.mozilla.org/projects/nss/raw-file/NSS_3_41_RTM/lib/ckfw/builtins/certdata.txt
```
The `_before` file will be used later. Verify that running `mk-ca-bundle`
made no changes to `src/node_root_certs.h`. If it did, something went wrong
with the previous update. Seek help!
Update metadata in the message below, and commit `certdata.txt`:
```text
tools: update certdata.txt
This is the certdata.txt[0] from NSS 3.41, released on 2018-12-03.
This is the version of NSS that will ship in Firefox 65 on
2018-12-11.
[0] https://hg.mozilla.org/projects/nss/raw-file/NSS_3_41_RTM/lib/ckfw/builtins/certdata.txt
```
3. Update `node_root_certs.h` from `certdata.txt`.
Run the command below:
```bash
./mk-ca-bundle.pl -v 2>_after
```
Confirm that `../src/node_root_certs.h` was updated.
Determine what changes were made by diffing the before and after files:
```console
% diff _before _after
11d10
< Parsing: Visa eCommerce Root
106d104
< Parsing: TÜRKTRUST Elektronik Sertifika Hizmet Sağlayıcısı H5
113,117d110
< Parsing: Certplus Root CA G1
< Parsing: Certplus Root CA G2
< Parsing: OpenTrust Root CA G1
< Parsing: OpenTrust Root CA G2
< Parsing: OpenTrust Root CA G3
134c127,136
< Done (133 CA certs processed, 20 skipped).
---
> Parsing: GlobalSign Root CA - R6
> Parsing: OISTE WISeKey Global Root GC CA
> Parsing: GTS Root R1
> Parsing: GTS Root R2
> Parsing: GTS Root R3
> Parsing: GTS Root R4
> Parsing: UCA Global G2 Root
> Parsing: UCA Extended Validation Root
> Parsing: Certigna Root CA
> Done (135 CA certs processed, 16 skipped).
```
Use the diff to update the message below, and commit `src/node_root_certs.h`:
```text
crypto: update root certificates
Update the list of root certificates in src/node_root_certs.h with
tools/mk-ca-bundle.pl.
Certificates added:
- GlobalSign Root CA - R6
- OISTE WISeKey Global Root GC CA
- GTS Root R1
- GTS Root R2
- GTS Root R3
- GTS Root R4
- UCA Global G2 Root
- UCA Extended Validation Root
- Certigna Root CA
Certificates removed:
- Visa eCommerce Root
- TÜRKTRUST Elektronik Sertifika Hizmet Sağlayıcısı H5
- Certplus Root CA G1
- Certplus Root CA G2
- OpenTrust Root CA G1
- OpenTrust Root CA G2
- OpenTrust Root CA G3
```
</details>
[NSS release schedule]: https://wiki.mozilla.org/NSS:Release_Versions
[tag list]: https://hg.mozilla.org/projects/nss/tags

View File

@ -0,0 +1,117 @@
# Maintaining shared library support
Node.js unofficially supports a build option where Node.js is built as
a shared library. The shared library is called libnode with additional postfixes
as appropriate for the platform (for example libnode.dll on windows).
The shared library provides a way to embed Node.js into other
applications and to have multiple applications use a single copy of
Node.js instead of having to bundle in the full Node.js footprint
into each application. For workloads that require multiple Node.js
instances this can result in a significant footprint savings.
This document provides an outline of the approach and things to look
out for when maintaining the shared library support.
Currently, shared library support has only been tested on:
* Linux
* macOS
* Windows
* AIX
## Building with shared library option
On non-Windows platforms, Node.js is built with the shared library
option by adding `--shared` to the configure step. On Windows
platforms Node.js is built with the shared library option by
adding `dll` to the vcbuild command line.
Once built there are two key components:
* executable - node
* library - libnode
The node executable is a thin wrapper around libnode which is
generated so that we can run the standard Node.js test suite
against the shared library.
The executable and library will have extensions as appropriate
for the platform on which they are built. For
example, node.exe on windows and node on other platforms for
the executable.
libnode may have additional naming components, as an example
in a build on macOS `libnode.105.dylib`. For non-windows platforms
the additional naming components include the `NODE_MODULE_VERSION` and
the appropriate postfix used for shared libraries on the platform.
In cases where an application links against the shared
library it is up to the application developer to add options
so that the shared library can be found by the application or
to set the LIBPATH (AIX), LD\_LIBRARY\_PATH (Linux/Unix), etc.
so that it is found at runtime.
For the node wrapper, on linux and macOS it is built
so that it can find the shared library in one of
the following:
* the same directory as the node executable
* ../lib with the expectation that the executable is
installed in a `bin` directory at the same level
as a `lib` directory in which the shared library is
installed. This is where the default package that
is build with the shared library option will
place the executable and library.
For the node wrapper on windows it is built expecting
that both the executable and shared library will
be in the same directory as it common practice on
that platform.
For the node wrapper on AIX, it is built with
the path to the shared library hardcoded as that
is the only option.
## Exports
On windows, functions that may be linked from native
addons or additional Node.js executables need to have
NODE\_EXTERN\_PRIVATE or NODE\_EXTERN otherwise they will
not be exported by the shared library. In the case of
functions used by additional Node.js executables
(ex: `mksnapshot`) a missing NODE\_EXTERN or
NODE\_EXTERN\_PRIVATE will cause the build to fail.
NODE\_EXTERN\_PRIVATE should be used in these cases
unless the intent is to add the function to the
public embedder API.
## Native addons
For regular Node.js builds, running native addons relies on symbols
exported by the node executable. As a result any
pre-built binaries expect symbols to be exported from the executable
instead of the shared library itself.
The node executable and shared library are built and linked
so that the required symbols are exported from the node
executable. This requires some extra work on some platforms
and the process to build the node executable is a good example
of how this can be achieved. Applications that use the shared
library and want to support native addons should employ
a similar technique.
## Testing
There is currently no testing in the regular CI run for PRs. There
are some CI jobs that can be used to test the shared library support and
some are run as part of daily testing. These include:
* [node-test-commit-linux-as-shared-lib](https://ci.nodejs.org/view/Node.js%20Daily/job/node-test-commit-linux-as-shared-lib/)
* <https://ci.nodejs.org/view/All/job/node-test-commit-osx-as-shared-lib/>
* [node-test-commit-aix-shared-lib](https://ci.nodejs.org/view/Node.js%20Daily/job/node-test-commit-aix-shared-lib/)
TODO: add a Job for windows
For code that modifies/affects shared library support these CI jobs should
run and it should be validated that there are no regressions in
the test suite.

View File

@ -0,0 +1,83 @@
# Maintaining Single Executable Applications support
Support for [single executable applications][] is one of the key technical
priorities identified for the success of Node.js.
## High level strategy
From the [Next-10 discussions][] there are 2 approaches the project believes are
important to support:
### Compile with Node.js into executable
This is the approach followed by [boxednode][].
No additional code within the Node.js project is needed to support the
option of compiling a bundled application along with Node.js into a single
executable application.
### Bundle into existing Node.js executable
This is the approach followed by [pkg][].
The project does not plan to provide the complete solution but instead the key
elements which are required in the Node.js executable in order to enable
bundling with the pre-built Node.js binaries. This includes:
* Looking for a segment within the executable that holds bundled code.
* Running the bundled code when such a segment is found.
It is left up to external tools/solutions to:
* Bundle code into a single script.
* Generate a command line with appropriate options.
* Add a segment to an existing Node.js executable which contains
the command line and appropriate headers.
* Re-generate or removing signatures on the resulting executable
* Provide a virtual file system, and hooking it in if needed to
support native modules or reading file contents.
However, the project also maintains a separate tool, [postject][], for injecting
arbitrary read-only resources into the binary such as those needed for bundling
the application into the runtime.
## Planning
Planning for this feature takes place in the [single-executable repository][].
## Upcoming features
Currently, only running a single embedded CommonJS file is supported but support
for the following features are in the list of work we'd like to get to:
* Running an embedded ESM file.
* Running an archive of multiple files.
* Embedding [Node.js CLI options][] into the binary.
* [XCOFF][] executable format.
* Run tests on Alpine Linux.
* Run tests on s390x Linux.
* Run tests on ppc64 Linux.
## Disabling single executable application support
To disable single executable application support, build Node.js with the
`--disable-single-executable-application` configuration option.
## Implementation
When built with single executable application support, the Node.js process uses
[`postject-api.h`][] to check if the `NODE_SEA_BLOB` section exists in the
binary. If it is found, it passes the buffer to
[`single_executable_application.js`][], which executes the contents of the
embedded script.
[Next-10 discussions]: https://github.com/nodejs/next-10/blob/main/meetings/summit-nov-2021.md#single-executable-applications
[Node.js CLI options]: https://nodejs.org/api/cli.html
[XCOFF]: https://www.ibm.com/docs/en/aix/7.2?topic=formats-xcoff-object-file-format
[`postject-api.h`]: https://github.com/nodejs/node/blob/71951a0e86da9253d7c422fa2520ee9143e557fa/test/fixtures/postject-copy/node_modules/postject/dist/postject-api.h
[`single_executable_application.js`]: https://github.com/nodejs/node/blob/main/lib/internal/main/single_executable_application.js
[boxednode]: https://github.com/mongodb-js/boxednode
[pkg]: https://github.com/vercel/pkg
[postject]: https://github.com/nodejs/postject
[single executable applications]: https://github.com/nodejs/node/blob/main/doc/contributing/technical-priorities.md#single-executable-applications
[single-executable repository]: https://github.com/nodejs/single-executable

View File

@ -0,0 +1,62 @@
# Maintaining the build files
This document explains how to maintain the build files in the codebase.
## Overview
On how to build the Node.js core, see [Building Node.js](../../../BUILDING.md).
There are three main build files that may be directly run when building Node.js:
* `configure`: A Python script that detects system capabilities and runs
[GYP][]. It generates `config.gypi` which includes parameters used by GYP to
create platform-dependent build files. Its output is usually in one of these
formats: Makefile, MSbuild, ninja, or XCode project files (the main
Makefile mentioned below is maintained separately by humans). For a detailed
guide on this script, see [configure][].
* `vcbuild.bat`: A Windows Batch Script that locates build tools, provides a
subset of the targets available in the [Makefile][], and a few
targets of its own. For a detailed guide on this script, see
[vcbuild.bat](#vcbuildbat).
* `Makefile`: A Makefile that can be run with GNU Make. It provides a set of
targets that build and test the Node.js binary, produce releases and
documentation, and interact with the CI to run benchmarks or tests. For a
detailed guide on this file, see [Makefile][].
On Windows `vcbuild.bat` runs [configure][] before building the
Node.js binary, on other systems `configure` must be run manually before running
`make` on the `Makefile`.
## vcbuild.bat
To see the help text, run `.\vcbuild help`. Update this file when you need to
update the build and testing process on Windows.
## configure
The `configure` script recognizes many CLI flags for special build formulas.
Many are not represented by `vcbuild` shortcuts, and need to be passed
either by:
* Calling `python configure --XXX --YYY=PPPP` directly, followed by `vcbuild
noprojgen`
* Setting `set config_flags=--XXX --YYY=PPPP` before calling `vcbuild`
To see the help text, run `python configure --help`. Update this file when you
need to update the configuration process.
## Makefile
To see the help text, run `make help`. This file is not generated, it is
maintained by humans. This is not usually run on Windows, where
[vcbuild.bat](#vcbuildbat) is used instead.
### Options
* `-j <n>`: number of threads used to build the binary. On the non-CI
targets, the parallel tests will take up all the available cores, regardless
of this option.
[GYP]: https://gyp.gsrc.io/docs/UserDocumentation.md
[Makefile]: #makefile
[configure]: #configure

View File

@ -0,0 +1,128 @@
# Maintaining types for Node.js
While JavaScript is a weakly-typed language, there are some complementary tools
like [TypeScript][] and [Flow][], which allow developers to annotate the source
code of their JavaScript projects. While many people don't annotate their code,
or make use of annotations at all, there are enough who do that the project has
agreed it's important to work towards having [suitable types for end-users][].
## High level approach - maintaining types
There are a number of ways that types could be maintained for Node.js ranging
from shipping them with the Node.js runtime to having them be externally
maintained.
The different options were discussed as part of the [next-10][] effort and it
was agreed that maintaining them externally is the best approach.
Some of the advantages to this approach include:
* Node.js maintainers do not need to be familiar with any given type
system/technology.
* Types can be updated without requiring Node.js releases.
The agreement was that the ideal flow would be as follows:
* APIs are added/documented in the existing Node.js markdown files.
* Automation in the Node.js project creates a machine readable JSON
representation of the API from the documentation.
* Automation within external type projects consumes the JSON and automatically
generates a PR to add the API.
## High level approach - development workflow
The number of people using TypeScript with Node.js is significant enough
that providing a good developer experience is important. While TypeScript
is identified specifically, a secondary goal is that what we provide to improve
development experience with TypeScript would apply to other type
systems and transpiled languages as well.
We have agreed that the approach will **NOT** include bundling TypeScript
tools with Node.js but instead improve the developer experience for how
those tools are installed/configured to work with Node.js.
The high level developer experience we are working towards was captured in the
[next-10 TypeScript mini-summit](https://github.com/nodejs/next-10/pull/150)
and is as follows:
1. When Node.js is started with an entry point that is not a file type that
Node.js recognizes, for example `node script.ts`, an informative error
message is printed that directs users to a webpage where they can
learn how to configure Node.js to support that file type.
* If the file was a TypeScript file, a TypeScript specific message with a
reference to a link on Nodejs.org specific on learning how to
configure TypeScript will be provided.
* For other file types a generic message and shared webpage will be
used.
2. Node.js gains support for loading configuration from a file. Most, if not
all, of the configuration supported by `NODE_OPTIONS` would be
supported in this file (which might be the `package.json` that lives
near the entry point file). The webpage with instructions would tell
users what configuration to put in this file to get Node.js to support
their file type.
3. When Node.js is run with the correct configuration, either in a file or
`NODE_OPTIONS` or flags, the unknown file type is executed as expected.
Some additional specifics around the current approach include:
* Loaders already provide a number of the components needed to
satisfy the requirements above. They already provide the Node.js
options that are needed to achieve many of the requirements above.
* `package.json` as the location for the config is potentially a good
choice as Node.js already looks for it as part of startup.
* The implementation chosen should allow for different configuration
in/for different environments/conditions such as production
versus development, or different types of hosted environments
such as serverless vs traditional, etc.; Node.js would not make
any recommendations or have any expectations as to what the
separate configuration blocks should be named or what their
purposes should be, just that a configuration file should have
the ability to provide different configurations for user-defined
conditions.
* There is no plan to define a default tsconfig.json for all Node.js users
* We don't have consensus on providing an opinionated default but
that should be explored after the initial steps are complete.
* It will be important that as part of the messaging around this
functionality that we avoid confusion that could lead people to ship
TypeScript files (e.g. `script.ts`) instead of the processed files
(e.g. `script.js`).
## Generation/Consumption of machine readable JSON files
When you run `make doc` the canonical markdown files used to
document the Node.js APIs in the
[doc/api][]
directory are converted to both an `.html` file and a `.json` file.
As part of the regular build/release process both the `html` and
`json` files are published to [nodejs.org][].
The generator that does the conversion is in the
[tools/doc][]
directory.
## Markdown structure
The constraints required on the markdown files in the
[doc/api][] directory
in order to be able to generate the JSON files are defined in the
[documentation-style-guide][].
## Planned changes (as of Jan 1 2022)
While JSON files are already being generated and published, they are not
structured well enough for them to be easily consumed by the type projects.
Generally external teams need some custom scripts along with manual fixup
afterwards.
There is an ongoing effort to add additional markdown constraints and
then update the flow in order to be able to generate a better
JSON output.
[Flow]: https://flow.org/
[TypeScript]: https://www.typescriptlang.org/
[doc/api]: https://github.com/nodejs/node/tree/HEAD/doc/api
[documentation-style-guide]: https://github.com/nodejs/node/blob/HEAD/doc/README.md#documentation-style-guide
[next-10]: https://github.com/nodejs/next-10/blob/HEAD/meetings/summit-nov-2021.md#suitable-types-for-end-users
[nodejs.org]: https://nodejs.org/en/docs/
[suitable types for end-users]: https://github.com/nodejs/node/blob/HEAD/doc/contributing/technical-priorities.md#suitable-types-for-end-users
[tools/doc]: https://github.com/nodejs/node/tree/HEAD/tools/doc

View File

@ -0,0 +1,90 @@
# Maintaining WebAssembly
Support for [WebAssembly](https://webassembly.org/)
has been identified as one of the
[top technical priorities](https://github.com/nodejs/node/blob/main/doc/contributing/technical-priorities.md#webassembly)
for the future success of Node.js.
This document provides an overview of our high-level strategy for
supporting WebAssembly and information about our current implementation
as a starting point for contributors.
## High-level approach
The key elements of our WebAssembly strategy include:
* Up-to-date core WebAssembly support
* Support for high-level APIs
* Making it easy to load WebAssembly
* Making sure the core Node.js APIs are compatible with WebAssembly
and can be called in an efficient manner from WebAssembly
### Up-to-date core WebAssembly support
Node.js gets its core WebAssembly support through V8. We don't need
to do anything specific to support this, all we have to do is keep
the version of V8 as up-to-date as possible.
### Key API support
As a runtime, Node.js must implement a number of APIs in addition
to the core WebAssembly support in order to be a good choice to run
WebAssembly. The project has currently identified these additional
APIs as important:
* WebAssembly System Interface (WASI). This provides the ability for
WebAssembly to interact with the outside world. Node.js currently
has an implementation (see below for more details).
* [WebAssembly Web API](https://www.w3.org/TR/wasm-web-api-1/). Node.js
currently has an implementation of streaming module compilation and
instantiation. As this and other specifications evolve, keeping up with them
will be important.
* [WebAssembly Component Model](https://github.com/WebAssembly/component-model/).
This API is still in the definition stage but the project should
keep track of its development as a way to simplify native code
integration.
### Making it as easy as possible to load WASM
The most important thing we can do on this front is to either find and
reference resources or provide resources on how to:
* Compile your WebAssembly code (outside of Node.js) and integrate that
into an npm workflow.
* Load and run WebAssembly code in your Node.js application.
It is also important to support and track the ongoing work in ESM to enable
loading of WebAssembly with ESM.
### Making sure the core Node.js APIs are compatible with WebAssembly
Use cases for which Node.js will be a good runtime will include code
both in JavaScript and compiled into WebAssembly. It is important
that Node.js APIs are able to be called from WebAssembly in
an efficient manner without extra buffer copies. We need to:
* Review APIs and identify those that can be called often from
WebAssembly.
* Where appropriate, make additions to identified APIs to allow
a pre-existing buffer to be passed in order to avoid copies.
## Current implementation and assets
### WebAssembly System Interface (WASI)
The Node.js WASI implementation is maintained in the
[uvwasi](https://github.com/nodejs/uvwasi) repository in the
Node.js GitHub organization. As needed, an updated copy
is vendored into the Node.js deps in
[deps/uvwasi](https://github.com/nodejs/node/tree/main/deps/uvwasi).
In addition to the code from uvwasi, Node.js includes bindings and
APIs that allow WebAssembly to be run with WASI support from Node.js.
The documentation for this API is in
[WebAssembly System Interface (WASI)](https://nodejs.org/api/wasi.html).
The implementation of the bindings and the public API is in:
* [src/node\_wasi.h](https://github.com/nodejs/node/blob/main/src/node_wasi.h)
* [src/node\_wasi.cc](https://github.com/nodejs/node/blob/main/src/node_wasi.cc)
* [lib/wasi.js](https://github.com/nodejs/node/blob/main/lib/wasi.js)

View File

@ -0,0 +1,49 @@
# Managing social media accounts
The project obtains and manages social media accounts in order to help
promote the project and to share content that benefits the ecosystem.
Currently the project has the following social media accounts:
* [bluesky](https://bsky.app/profile/nodejs.org)
* [twitter](https://x.com/nodejs)
* [Facebook](https://www.facebook.com/nodejsfoundation)
* [LinkedIn](https://www.linkedin.com/company/node.js-foundation/)
## bluesky
Bluesky is managed directly by the project. Automation is used
to request posts/reposts as outlined in
[nodejs/bluesky](https://github.com/nodejs/bluesky).
Login credentials are available to members of the Technical
Steering Committee, but are not to be used directly except
in exceptional circumstances.
## Twitter/X
Day to day management of the Twitter/X account has been
delegated to the Foundation Staff.
In addition, it has been agreed that Releasers may login
and post when doing a release, and TSC members may post
an announcement of a release if a releaser is not able to
do it themselves.
Login credentials are available to members of the Technical
Steering Committee. Releasers may request access by
opening an issue in the nodejs/admin repo asking for access.
## Facebook
Day to day management of the Facebook account has been
delegated to the Foundation Staff.
No project members have access to the Facebook account.
## Linkedin
Day to day management of the LinkedIn account has been
delegated to the Foundation Staff.
No project members have access to the LinkedIn account.

View File

@ -0,0 +1,72 @@
# Postmortem support
Postmortem metadata are constants present in the final build which can be used
by debuggers and other tools to navigate through internal structures of software
when analyzing its memory (either on a running process or a core dump). Node.js
provides this metadata in its builds for V8 and Node.js internal structures.
## V8 postmortem metadata
V8 prefixes all postmortem constants with `v8dbg_`, and they allow inspection of
objects on the heap as well as object properties and references. V8 generates
those symbols with a script (`deps/v8/tools/gen-postmortem-metadata.py`), and
Node.js always includes these constants in the final build.
## Node.js debug symbols
Node.js prefixes all postmortem constants with `nodedbg_`, and they complement
V8 constants by providing ways to inspect Node.js-specific structures, like
`node::Environment`, `node::BaseObject` and its descendants, classes from
`src/utils.h` and others. Those constants are declared in
`src/node_postmortem_metadata.cc`, and most of them are calculated at compile
time.
### Calculating offset of class members
Node.js constants referring to the offset of class members in memory
are calculated at compile time.
Because of that, those class members must be at a fixed offset
from the start of the class. That's not a problem in most cases, but it also
means that those members should always come after any templated member on the
class definition.
For example, if we want to add a constant with the offset for
`ReqWrap::req_wrap_queue_`, it should be defined after `ReqWrap::req_`, because
`sizeof(req_)` depends on the type of T, which means the class definition should
be like this:
```cpp
template <typename T>
class ReqWrap : public AsyncWrap {
private:
// req_wrap_queue_ comes before any templated member, which places it in a
// fixed offset from the start of the class
ListNode<ReqWrap> req_wrap_queue_;
T req_;
};
```
instead of:
```cpp
template <typename T>
class ReqWrap : public AsyncWrap {
private:
T req_;
// req_wrap_queue_ comes after a templated member, which means it won't be in
// a fixed offset from the start of the class
ListNode<ReqWrap> req_wrap_queue_;
};
```
There are also tests on `test/cctest/test_node_postmortem_metadata.cc` to make
sure all Node.js postmortem metadata are calculated correctly.
## Tools and references
* [llnode](https://github.com/nodejs/llnode): LLDB plugin
* [`mdb_v8`](https://github.com/joyent/mdb_v8): mdb plugin
* [nodejs/post-mortem](https://github.com/nodejs/post-mortem): Node.js
post-mortem working group

View File

@ -0,0 +1,25 @@
# Offboarding
This document is a checklist of things to do when a collaborator becomes
emeritus or leaves the project.
* Remove the collaborator from the [`@nodejs/collaborators`][] team.
* Open a fast-track pull request to move the collaborator to the collaborator
emeriti list in README.md.
* Determine what GitHub teams the collaborator belongs to. In consultation with
the collaborator, determine which of those teams they should be removed from.
* Some teams may also require a pull request to remove the collaborator from
a team listing. For example, if someone is removed from @nodejs/build,
they should also be removed from the Build WG README.md file in the
<https://github.com/nodejs/build> repository.
* When in doubt, especially if you are unable to get in contact with the
collaborator, remove them from all teams. It is easy enough to add them
back later, so we err on the side of privacy and security.
* Remove them from the [`@nodejs`](https://github.com/orgs/nodejs/people) GitHub
org unless they are members for a reason other than being a Collaborator.
* [Open an issue](https://github.com/nodejs/build/issues/new) in the
nodejs/build repository titled `Remove Collaborator from Coverity` asking that
the collaborator be removed from the Node.js coverity project if they had
access.
[`@nodejs/collaborators`]: https://github.com/orgs/nodejs/teams/collaborators/members

View File

@ -0,0 +1,836 @@
# Usage of primordials in core
The file `lib/internal/per_context/primordials.js` subclasses and stores the JS
built-ins that come from the VM so that Node.js built-in modules do not need to
later look these up from the global proxy, which can be mutated by users.
For some area of the codebase, performance and code readability are deemed more
important than reliability against prototype pollution:
* `node:http`
* `node:http2`
* `node:tls`
* `node:zlib`
Usage of primordials should be preferred for new code in other areas, but
replacing current code with primordials should be
[done with care](#primordials-with-known-performance-issues). It is highly
recommended to ping the relevant team when reviewing a pull request that touches
one of the subsystems they "own".
## Accessing primordials
The primordials are meant for internal use only, and are only accessible for
internal core modules. User code cannot use or rely on primordials. It is
usually fine to rely on ECMAScript built-ins and assume that it will behave as
specified.
If you would like to access the `primordials` object to help you with Node.js
core development or for tinkering, you can expose it on the global scope using
this combination of CLI flags:
```bash
node --expose-internals -r internal/test/binding
```
## Contents of primordials
### Properties of the global object
Objects and functions on the global object can be deleted or replaced. Using
them from primordials makes the code more reliable:
```js
globalThis.Array === primordials.Array; // true
globalThis.Array = function() {
return [1, 2, 3];
};
globalThis.Array === primordials.Array; // false
primordials.Array(0); // []
globalThis.Array(0); // [1,2,3]
```
### Prototype methods
ECMAScript provides a group of methods available on built-in objects that are
used to interact with JavaScript objects.
```js
const array = [1, 2, 3];
array.push(4); // Here `push` refers to %Array.prototype.push%.
console.log(JSON.stringify(array)); // [1,2,3,4]
// %Array.prototype%.push is modified in userland.
Array.prototype.push = function push(val) {
return this.unshift(val);
};
array.push(5); // Now `push` refers to the modified method.
console.log(JSON.stringify(array)); // [5,1,2,3,4]
```
Primordials wrap the original prototype functions with new functions that take
the `this` value as the first argument:
```js
const {
ArrayPrototypePush,
} = primordials;
const array = [1, 2, 3];
ArrayPrototypePush(array, 4);
console.log(JSON.stringify(array)); // [1,2,3,4]
Array.prototype.push = function push(val) {
return this.unshift(val);
};
ArrayPrototypePush(array, 5);
console.log(JSON.stringify(array)); // [1,2,3,4,5]
```
### Safe classes
Safe classes are classes that provide the same API as their equivalent class,
but whose implementation aims to avoid any reliance on user-mutable code.
Safe classes should not be exposed to user-land; use unsafe equivalent when
dealing with objects that are accessible from user-land.
### Variadic functions
There are some built-in functions that accept a variable number of arguments
(e.g.: `Math.max`, `%Array.prototype.push%`). It is sometimes useful to provide
the list of arguments as an array. You can use primordial function with the
suffix `Apply` (e.g.: `MathMaxApply`, `ArrayPrototypePushApply`) to do that.
## Primordials with known performance issues
One of the reasons why the current Node.js API is not completely tamper-proof is
performance: sometimes the use of primordials can cause performance regressions
with V8, which when in a hot code path, could significantly decrease the
performance of code in Node.js.
* Methods that mutate the internal state of arrays:
* `ArrayPrototypePush`
* `ArrayPrototypePop`
* `ArrayPrototypeShift`
* `ArrayPrototypeUnshift`
* Methods of the function prototype:
* `FunctionPrototypeBind`
* `FunctionPrototypeCall`: creates performance issues when used to invoke
super constructors.
* `FunctionPrototype`: use `() => {}` instead when referencing a no-op
function.
* `SafeArrayIterator`
* `SafeStringIterator`
* `SafePromiseAll`
* `SafePromiseAllSettled`
* `SafePromiseAny`
* `SafePromiseRace`
* `SafePromisePrototypeFinally`: use `try {} finally {}` block instead.
* `ReflectConstruct`: Also affects `Reflect.construct`.
`ReflectConstruct` creates new types of classes inside functions.
Instead consider creating a shared class. See [nodejs/performance#109](https://github.com/nodejs/performance/issues/109).
In general, when sending or reviewing a PR that makes changes in a hot code
path, use extra caution and run extensive benchmarks.
## Implicit use of user-mutable methods
### Unsafe array iteration
There are many usual practices in JavaScript that rely on iteration. It's useful
to be aware of them when dealing with arrays (or `TypedArray`s) in core as array
iteration typically calls several user-mutable methods. This sections lists the
most common patterns in which ECMAScript code relies non-explicitly on array
iteration and how to avoid it.
<details>
<summary>Avoid for-of loops on arrays</summary>
```js
for (const item of array) {
console.log(item);
}
```
This code is internally expanded into something that looks like:
```js
{
// 1. Lookup %Symbol.iterator% property on `array` (user-mutable if
// user-provided).
// 2. Lookup %Symbol.iterator% property on %Array.prototype% (user-mutable).
// 3. Call that function.
const iterator = array[Symbol.iterator]();
// 1. Lookup `next` property on `iterator` (doesn't exist).
// 2. Lookup `next` property on %ArrayIteratorPrototype% (user-mutable).
// 3. Call that function.
let { done, value: item } = iterator.next();
while (!done) {
console.log(item);
// Repeat.
({ done, value: item } = iterator.next());
}
}
```
Instead of utilizing iterators, you can use the more traditional but still very
performant `for` loop:
```js
for (let i = 0; i < array.length; i++) {
console.log(array[i]);
}
```
The following code snippet illustrates how user-land code could impact the
behavior of internal modules:
```js
// User-land
Array.prototype[Symbol.iterator] = () => ({
next: () => ({ done: true }),
});
// Core
let forOfLoopBlockExecuted = false;
let forLoopBlockExecuted = false;
const array = [1, 2, 3];
for (const item of array) {
forOfLoopBlockExecuted = true;
}
for (let i = 0; i < array.length; i++) {
forLoopBlockExecuted = true;
}
console.log(forOfLoopBlockExecuted); // false
console.log(forLoopBlockExecuted); // true
```
This only applies if you are working with a genuine array (or array-like
object). If you are instead expecting an iterator, a for-of loop may be a better
choice.
</details>
<details>
<summary>Avoid array destructuring assignment on arrays</summary>
```js
const [first, second] = array;
```
This is roughly equivalent to:
```js
// 1. Lookup %Symbol.iterator% property on `array` (user-mutable if
// user-provided).
// 2. Lookup %Symbol.iterator% property on %Array.prototype% (user-mutable).
// 3. Call that function.
const iterator = array[Symbol.iterator]();
// 1. Lookup `next` property on `iterator` (doesn't exist).
// 2. Lookup `next` property on %ArrayIteratorPrototype% (user-mutable).
// 3. Call that function.
const first = iterator.next().value;
// Repeat.
const second = iterator.next().value;
```
Instead you can use object destructuring:
```js
const { 0: first, 1: second } = array;
```
or
```js
const first = array[0];
const second = array[1];
```
This only applies if you are working with a genuine array (or array-like
object). If you are instead expecting an iterator, array destructuring is the
best choice.
</details>
<details>
<summary>Avoid spread operator on arrays</summary>
```js
// 1. Lookup %Symbol.iterator% property on `array` (user-mutable if
// user-provided).
// 2. Lookup %Symbol.iterator% property on %Array.prototype% (user-mutable).
// 3. Lookup `next` property on %ArrayIteratorPrototype% (user-mutable).
const arrayCopy = [...array];
func(...array);
```
Instead you can use other ECMAScript features to achieve the same result:
```js
const arrayCopy = ArrayPrototypeSlice(array);
ReflectApply(func, null, array);
```
</details>
<details>
<summary><code>%Array.prototype.concat%</code> looks up
<code>%Symbol.isConcatSpreadable%</code> property of the passed
arguments and the <code>this</code> value</summary>
```js
{
// Unsafe code example:
// 1. Lookup %Symbol.isConcatSpreadable% property on `array`
// (user-mutable if user-provided).
// 2. Lookup %Symbol.isConcatSpreadable% property on `%Array.prototype%
// (user-mutable).
// 2. Lookup %Symbol.isConcatSpreadable% property on `%Object.prototype%
// (user-mutable).
const array = [];
ArrayPrototypeConcat(array);
}
```
```js
// User-land
Object.defineProperty(Object.prototype, Symbol.isConcatSpreadable, {
get() {
this.push(5);
return true;
},
});
// Core
{
// Using ArrayPrototypeConcat does not produce the expected result:
const a = [1, 2];
const b = [3, 4];
console.log(ArrayPrototypeConcat(a, b)); // [1, 2, 5, 3, 4, 5]
}
{
// Concatenating two arrays can be achieved safely, e.g.:
const a = [1, 2];
const b = [3, 4];
// Using %Array.prototype.push% and `SafeArrayIterator` to get the expected
// outcome:
const concatArray = [];
ArrayPrototypePush(concatArray, ...new SafeArrayIterator(a),
...new SafeArrayIterator(b));
console.log(concatArray); // [1, 2, 3, 4]
// Or using `ArrayPrototypePushApply` if it's OK to mutate the first array:
ArrayPrototypePushApply(a, b);
console.log(a); // [1, 2, 3, 4]
}
```
</details>
<details>
<summary><code>%Object.fromEntries%</code> iterate over an array</summary>
```js
{
// Unsafe code example:
// 1. Lookup %Symbol.iterator% property on `array` (user-mutable if
// user-provided).
// 2. Lookup %Symbol.iterator% property on %Array.prototype% (user-mutable).
// 3. Lookup `next` property on %ArrayIteratorPrototype% (user-mutable).
const obj = ObjectFromEntries(array);
}
{
// Safe example using `SafeArrayIterator`:
const obj = ObjectFromEntries(new SafeArrayIterator(array));
}
{
// Safe example without using `SafeArrayIterator`:
const obj = {};
for (let i = 0; i < array.length; i++) {
obj[array[i][0]] = array[i][1];
}
// In a hot code path, this would be the preferred method.
}
```
</details>
<details>
<summary><code>%Promise.all%</code>,
<code>%Promise.allSettled%</code>,
<code>%Promise.any%</code>, and
<code>%Promise.race%</code> iterate over an array</summary>
```js
// 1. Lookup %Symbol.iterator% property on `array` (user-mutable if
// user-provided).
// 2. Lookup %Symbol.iterator% property on %Array.prototype% (user-mutable).
// 3. Lookup `next` property on %ArrayIteratorPrototype% (user-mutable).
// 4. Lookup `then` property on %Array.Prototype% (user-mutable).
// 5. Lookup `then` property on %Object.Prototype% (user-mutable).
PromiseAll([]); // unsafe
// 1. Lookup `then` property on %Array.Prototype% (user-mutable).
// 2. Lookup `then` property on %Object.Prototype% (user-mutable).
PromiseAll(new SafeArrayIterator([])); // still unsafe
SafePromiseAll([]); // still unsafe
SafePromiseAllReturnVoid([]); // safe
SafePromiseAllReturnArrayLike([]); // safe
const array = [promise];
const set = new SafeSet().add(promise);
// When running one of these functions on a non-empty iterable, it will also:
// 1. Lookup `then` property on `promise` (user-mutable if user-provided).
// 2. Lookup `then` property on `%Promise.prototype%` (user-mutable).
// 3. Lookup `then` property on %Array.Prototype% (user-mutable).
// 4. Lookup `then` property on %Object.Prototype% (user-mutable).
PromiseAll(new SafeArrayIterator(array)); // unsafe
PromiseAll(set); // unsafe
SafePromiseAllReturnVoid(array); // safe
SafePromiseAllReturnArrayLike(array); // safe
// Some key differences between `SafePromise[...]` and `Promise[...]` methods:
// 1. SafePromiseAll, SafePromiseAllSettled, SafePromiseAny, SafePromiseRace,
// SafePromiseAllReturnArrayLike, SafePromiseAllReturnVoid, and
// SafePromiseAllSettledReturnVoid support passing a mapperFunction as second
// argument.
SafePromiseAll(ArrayPrototypeMap(array, someFunction));
SafePromiseAll(array, someFunction); // Same as the above, but more efficient.
// 2. SafePromiseAll, SafePromiseAllSettled, SafePromiseAny, SafePromiseRace,
// SafePromiseAllReturnArrayLike, SafePromiseAllReturnVoid, and
// SafePromiseAllSettledReturnVoid only support arrays and array-like
// objects, not iterables. Use ArrayFrom to convert an iterable to an array.
SafePromiseAllReturnVoid(set); // ignores set content.
SafePromiseAllReturnVoid(ArrayFrom(set)); // works
// 3. SafePromiseAllReturnArrayLike is safer than SafePromiseAll, however you
// should not use them when its return value is passed to the user as it can
// be surprising for them not to receive a genuine array.
SafePromiseAllReturnArrayLike(array).then((val) => val instanceof Array); // false
SafePromiseAll(array).then((val) => val instanceof Array); // true
```
</details>
<details>
<summary><code>%Map%</code>, <code>%Set%</code>, <code>%WeakMap%</code>, and
<code>%WeakSet%</code> constructors iterate over an array</summary>
```js
// User-land
Array.prototype[Symbol.iterator] = () => ({
next: () => ({ done: true }),
});
// Core
// 1. Lookup %Symbol.iterator% property on %Array.prototype% (user-mutable).
// 2. Lookup `next` property on %ArrayIteratorPrototype% (user-mutable).
const set = new SafeSet([1, 2, 3]);
console.log(set.size); // 0
```
```js
// User-land
Array.prototype[Symbol.iterator] = () => ({
next: () => ({ done: true }),
});
// Core
const set = new SafeSet();
set.add(1).add(2).add(3);
console.log(set.size); // 3
```
</details>
### Promise objects
<details>
<summary><code>%Promise.prototype.finally%</code> looks up <code>then</code>
property of the Promise instance</summary>
```js
// User-land
Promise.prototype.then = function then(a, b) {
return Promise.resolve();
};
// Core
let finallyBlockExecuted = false;
PromisePrototypeFinally(somePromiseThatEventuallySettles,
() => { finallyBlockExecuted = true; });
process.on('exit', () => console.log(finallyBlockExecuted)); // false
```
```js
// User-land
Promise.prototype.then = function then(a, b) {
return Promise.resolve();
};
// Core
let finallyBlockExecuted = false;
(async () => {
try {
return await somePromiseThatEventuallySettles;
} finally {
finallyBlockExecuted = true;
}
})();
process.on('exit', () => console.log(finallyBlockExecuted)); // true
```
</details>
<details>
<summary><code>%Promise.all%</code>,
<code>%Promise.allSettled%</code>,
<code>%Promise.any%</code>, and
<code>%Promise.race%</code> look up <code>then</code>
property of the Promise instances</summary>
You can use safe alternatives from primordials that differ slightly from the
original methods:
* It expects an array (or array-like object) instead of an iterable.
* It wraps each promise in `SafePromise` objects and wraps the result in a new
`Promise` instance – which may come with a performance penalty.
* It accepts a `mapperFunction` as second argument.
* Because it doesn't look up `then` property, it may not be the right tool to
handle user-provided promises (which may be instances of a subclass of
`Promise`).
```js
// User-land
Promise.prototype.then = function then(a, b) {
return Promise.resolve();
};
// Core
let thenBlockExecuted = false;
PromisePrototypeThen(
PromiseAll(new SafeArrayIterator([PromiseResolve()])),
() => { thenBlockExecuted = true; },
);
process.on('exit', () => console.log(thenBlockExecuted)); // false
```
```js
// User-land
Promise.prototype.then = function then(a, b) {
return Promise.resolve();
};
// Core
let thenBlockExecuted = false;
PromisePrototypeThen(
SafePromiseAll([PromiseResolve()]),
() => { thenBlockExecuted = true; },
);
process.on('exit', () => console.log(thenBlockExecuted)); // true
```
A common pattern is to map on the array of `Promise`s to apply some
transformations, in that case it can be more efficient to pass a second argument
rather than invoking `%Array.prototype.map%`.
```js
SafePromiseAll(ArrayPrototypeMap(array, someFunction));
SafePromiseAll(array, someFunction); // Same as the above, but more efficient.
```
</details>
### (Async) Generator functions
Generators and async generators returned by generator functions and async
generator functions are relying on user-mutable methods; their use in core
should be avoided.
<details>
<summary><code>%GeneratorFunction.prototype.prototype%.next</code> is
user-mutable</summary>
```js
// User-land
Object.getPrototypeOf(function* () {}).prototype.next = function next() {
return { done: true };
};
// Core
function* someGenerator() {
yield 1;
yield 2;
yield 3;
}
let loopCodeExecuted = false;
for (const nb of someGenerator()) {
loopCodeExecuted = true;
}
console.log(loopCodeExecuted); // false
```
</details>
<details>
<summary><code>%AsyncGeneratorFunction.prototype.prototype%.next</code> is
user-mutable</summary>
```js
// User-land
Object.getPrototypeOf(async function* () {}).prototype.next = function next() {
return new Promise(() => {});
};
// Core
async function* someGenerator() {
yield 1;
yield 2;
yield 3;
}
let finallyBlockExecuted = false;
async () => {
try {
for await (const nb of someGenerator()) {
// some code;
}
} finally {
finallyBlockExecuted = true;
}
};
process.on('exit', () => console.log(finallyBlockExecuted)); // false
```
</details>
### Text processing
#### Unsafe string methods
| The string method | looks up the property |
| ----------------------------- | --------------------- |
| `String.prototype.match` | `Symbol.match` |
| `String.prototype.matchAll` | `Symbol.matchAll` |
| `String.prototype.replace` | `Symbol.replace` |
| `String.prototype.replaceAll` | `Symbol.replace` |
| `String.prototype.search` | `Symbol.search` |
| `String.prototype.split` | `Symbol.split` |
```js
// User-land
RegExp.prototype[Symbol.replace] = () => 'foo';
String.prototype[Symbol.replace] = () => 'baz';
// Core
console.log(StringPrototypeReplace('ber', /e/, 'a')); // 'foo'
console.log(StringPrototypeReplace('ber', 'e', 'a')); // 'baz'
console.log(RegExpPrototypeSymbolReplace(/e/, 'ber', 'a')); // 'bar'
```
#### Unsafe string iteration
As with arrays, iterating over strings calls several user-mutable methods. Avoid
iterating over strings when possible, or use `SafeStringIterator`.
#### Unsafe `RegExp` methods
Functions that lookup the `exec` property on the prototype chain:
* `RegExp.prototype[Symbol.match]`
* `RegExp.prototype[Symbol.matchAll]`
* `RegExp.prototype[Symbol.replace]`
* `RegExp.prototype[Symbol.search]`
* `RegExp.prototype[Symbol.split]`
* `RegExp.prototype.test`
```js
// User-land
RegExp.prototype.exec = () => null;
// Core
console.log(RegExpPrototypeTest(/o/, 'foo')); // false
console.log(RegExpPrototypeExec(/o/, 'foo') !== null); // true
console.log(RegExpPrototypeSymbolSearch(/o/, 'foo')); // -1
console.log(SafeStringPrototypeSearch('foo', /o/)); // 1
```
#### Don't trust `RegExp` flags
RegExp flags are not own properties of the regex instances, which means flags
can be reset from user-land.
<details>
<summary>List of <code>RegExp</code> methods that look up properties from
mutable getters</summary>
| `RegExp` method | looks up the following flag-related properties |
| ----------------------------------- | ------------------------------------------------------------------ |
| `get RegExp.prototype.flags` | `global`, `ignoreCase`, `multiline`, `dotAll`, `unicode`, `sticky` |
| `RegExp.prototype[Symbol.match]` | `global`, `unicode` |
| `RegExp.prototype[Symbol.matchAll]` | `flags` |
| `RegExp.prototype[Symbol.replace]` | `global`, `unicode` |
| `RegExp.prototype[Symbol.split]` | `flags` |
| `RegExp.prototype.toString` | `flags` |
</details>
```js
// User-land
Object.defineProperty(RegExp.prototype, 'global', { value: false });
// Core
console.log(RegExpPrototypeSymbolReplace(/o/g, 'foo', 'a')); // 'fao'
console.log(RegExpPrototypeSymbolReplace(hardenRegExp(/o/g), 'foo', 'a')); // 'faa'
```
### Defining object own properties
When defining property descriptor (to add or update an own property to a
JavaScript object), be sure to always use a null-prototype object to avoid
prototype pollution.
```js
// User-land
Object.prototype.get = function get() {};
// Core
try {
ObjectDefineProperty({}, 'someProperty', { value: 0 });
} catch (err) {
console.log(err); // TypeError: Invalid property descriptor.
}
```
```js
// User-land
Object.prototype.get = function get() {};
// Core
ObjectDefineProperty({}, 'someProperty', { __proto__: null, value: 0 });
console.log('no errors'); // no errors.
```
Same applies when trying to modify an existing property, e.g. trying to make a
read-only property enumerable:
```js
// User-land
Object.prototype.value = 'Unrelated user-provided data';
// Core
class SomeClass {
get readOnlyProperty() { return 'genuine data'; }
}
ObjectDefineProperty(SomeClass.prototype, 'readOnlyProperty', { enumerable: true });
console.log(new SomeClass().readOnlyProperty); // Unrelated user-provided data
```
```js
// User-land
Object.prototype.value = 'Unrelated user-provided data';
// Core
const kEnumerableProperty = { __proto__: null, enumerable: true };
// In core, use const {kEnumerableProperty} = require('internal/util');
class SomeClass {
get readOnlyProperty() { return 'genuine data'; }
}
ObjectDefineProperty(SomeClass.prototype, 'readOnlyProperty', kEnumerableProperty);
console.log(new SomeClass().readOnlyProperty); // genuine data
```
### Defining a `Proxy` handler
When defining a `Proxy`, the handler object could be at risk of prototype
pollution when using a plain object literal:
```js
// User-land
Object.prototype.get = () => 'Unrelated user-provided data';
// Core
const objectToProxy = { someProperty: 'genuine value' };
const proxyWithPlainObjectLiteral = new Proxy(objectToProxy, {
has() { return false; },
});
console.log(proxyWithPlainObjectLiteral.someProperty); // Unrelated user-provided data
const proxyWithNullPrototypeObject = new Proxy(objectToProxy, {
__proto__: null,
has() { return false; },
});
console.log(proxyWithNullPrototypeObject.someProperty); // genuine value
```
### Checking if an object is an instance of a class
#### Using `instanceof` looks up the `%Symbol.hasInstance%` property of the class
```js
// User-land
Object.defineProperty(Array, Symbol.hasInstance, {
__proto__: null,
value: () => true,
});
Object.defineProperty(Date, Symbol.hasInstance, {
__proto__: null,
value: () => false,
});
// Core
const {
FunctionPrototypeSymbolHasInstance,
} = primordials;
console.log(new Date() instanceof Array); // true
console.log(new Date() instanceof Date); // false
console.log(FunctionPrototypeSymbolHasInstance(Array, new Date())); // false
console.log(FunctionPrototypeSymbolHasInstance(Date, new Date())); // true
```
Even without user mutations, the result of `instanceof` can be deceiving when
dealing with values from different realms:
```js
const vm = require('node:vm');
console.log(vm.runInNewContext('[]') instanceof Array); // false
console.log(vm.runInNewContext('[]') instanceof vm.runInNewContext('Array')); // false
console.log([] instanceof vm.runInNewContext('Array')); // false
console.log(Array.isArray(vm.runInNewContext('[]'))); // true
console.log(vm.runInNewContext('Array').isArray(vm.runInNewContext('[]'))); // true
console.log(vm.runInNewContext('Array').isArray([])); // true
```
In general, using `instanceof` (or `FunctionPrototypeSymbolHasInstance`) checks
is not recommended, consider checking for the presence of properties or methods
for more reliable results.

View File

@ -0,0 +1,607 @@
# Pull requests
* [Dependencies](#dependencies)
* [Setting up your local environment](#setting-up-your-local-environment)
* [Step 1: Fork](#step-1-fork)
* [Step 2: Branch](#step-2-branch)
* [The process of making changes](#the-process-of-making-changes)
* [Step 3: Code](#step-3-code)
* [Step 4: Commit](#step-4-commit)
* [Commit message guidelines](#commit-message-guidelines)
* [Step 5: Rebase](#step-5-rebase)
* [Step 6: Test](#step-6-test)
* [Step 7: Push](#step-7-push)
* [Step 8: Opening the pull request](#step-8-opening-the-pull-request)
* [Step 9: Discuss and update](#step-9-discuss-and-update)
* [Approval and request changes workflow](#approval-and-request-changes-workflow)
* [Step 10: Landing](#step-10-landing)
* [Reviewing pull requests](#reviewing-pull-requests)
* [Review a bit at a time](#review-a-bit-at-a-time)
* [Be aware of the person behind the code](#be-aware-of-the-person-behind-the-code)
* [Respect the minimum wait time for comments](#respect-the-minimum-wait-time-for-comments)
* [Abandoned or stalled pull requests](#abandoned-or-stalled-pull-requests)
* [Approving a change](#approving-a-change)
* [Accept that there are different opinions about what belongs in Node.js](#accept-that-there-are-different-opinions-about-what-belongs-in-nodejs)
* [Performance is not everything](#performance-is-not-everything)
* [Continuous integration testing](#continuous-integration-testing)
* [Notes](#notes)
* [Commit squashing](#commit-squashing)
* [Getting approvals for your pull request](#getting-approvals-for-your-pull-request)
* [Waiting until the pull request gets landed](#waiting-until-the-pull-request-gets-landed)
* [Check out the collaborator guide](#check-out-the-collaborator-guide)
* [Appendix: subsystems](#appendix-subsystems)
## Dependencies
Node.js has several bundled dependencies in the _deps/_ and the _tools/_
directories that are not part of the project proper.
These are detailed in the [maintaining dependencies][] document.
Changes to files in those directories should be sent
to their respective projects.
Do not send a patch to Node.js. We cannot accept such patches.
In case of doubt, open an issue in the
[issue tracker](https://github.com/nodejs/node/issues/) or contact one of the
[project collaborators](https://github.com/nodejs/node/#current-project-team-members).
Node.js has many channels on the
[OpenJS Foundation Slack](https://slack-invite.openjsf.org/). Interesting
channels are:
[#nodejs](https://openjs-foundation.slack.com/archives/CK9Q4MB53) for general
help, questions, and discussions.
[#nodejs-core](https://openjs-foundation.slack.com/archives/C019Y2T6STH) for
development of Node.js core specifically.
Node.js also has an unofficial IRC channel:
[#Node.js](https://web.libera.chat/#node.js).
## Setting up your local environment
To get started, you will need to have `git` installed locally. Depending on
your operating system, there are also a number of other dependencies required.
These are detailed in the [Building guide][].
Depending on your environment you might want to grab IDE specific settings from
[IDE configs](https://github.com/nodejs/node-code-ide-configs).
Once you have `git` and are sure you have all of the necessary dependencies,
it's time to create a fork.
### Step 1: Fork
Fork the project [on GitHub](https://github.com/nodejs/node) and clone your fork
locally.
```bash
git clone git@github.com:username/node.git
cd node
git remote add upstream https://github.com/nodejs/node.git
git fetch upstream
```
Configure `git` so that it knows who you are:
```bash
git config user.name "J. Random User"
git config user.email "j.random.user@example.com"
```
You can use any name/email address you prefer here. We only use the
metadata generated by `git` using this configuration for properly attributing
your changes to you in the `AUTHORS` file and the changelog.
If you would like for the GitHub UI to link the commit to your account
and award you the `Contributor` label after the changes have been merged,
make sure this local email is also added to your
[GitHub email list](https://github.com/settings/emails).
### Step 2: Branch
As a best practice to keep your development environment as organized as
possible, create local branches to work within. These should also be created
directly off of the upstream default branch.
```bash
git checkout -b my-branch -t upstream/HEAD
```
## The process of making changes
### Step 3: Code
Pull requests in Node.js typically involve changes to
one or more of a few places in the repository.
* C/C++ code contained in the `src` directory
* JavaScript code contained in the `lib` directory
* Documentation in `doc/api`
* Tests within the `test` directory
If you are modifying code, please be sure to run `make lint` (or
`vcbuild.bat lint` on Windows) to ensure that the changes follow the Node.js
code style guide.
Any documentation you write (including code comments and API documentation)
should follow the [Style Guide](../../doc/README.md). Code samples
included in the API docs will also be checked when running `make lint` (or
`vcbuild.bat lint` on Windows). If you are adding to or deprecating an API,
add or change the appropriate YAML documentation. Use `REPLACEME` for the
version number in the documentation YAML:
```markdown
### `request.method`
<!-- YAML
added: REPLACEME
-->
* {string} The request method.
```
For contributing C++ code, you may want to look at the
[C++ Style Guide](cpp-style-guide.md), as well as the
[README of `src/`](../../src/README.md) for an overview of Node.js
C++ internals.
### Step 4: Commit
It is a best practice to keep your changes as logically grouped
as possible within individual commits. There is no limit to the number of
commits any single pull request may have, and many contributors find it easier
to review changes that are split across multiple commits.
```bash
git add my/changed/files
git commit
```
Multiple commits often get squashed when they are landed. See the
notes about [commit squashing](#commit-squashing).
#### Commit message guidelines
A good commit message should describe what changed and why.
1. The first line should:
* contain a short description of the change (preferably 50 characters or
less, and no more than 72 characters)
* be entirely in lowercase with the exception of proper nouns, acronyms, and
the words that refer to code, like function/variable names
* be prefixed with the name of the changed [subsystem](#appendix-subsystems)
and start with an imperative verb. Check the output of `git log --oneline
files/you/changed` to find out what subsystems your changes touch.
Examples:
* `net: add localAddress and localPort to Socket`
* `src: fix typos in async_wrap.h`
2. Keep the second line blank.
3. Wrap all other lines at 72 columns (except for long URLs).
4. If your patch fixes an open issue, you can add a reference to it at the end
of the log. Use the `Fixes:` prefix and the full issue URL. For other
references use `Refs:`.
`Fixes:` and `Refs:` trailers get automatically added to your commit message
when the Pull Request lands as long as they are included in the
Pull Request's description. If the Pull Request lands in several commits,
by default the trailers found in the description are added to each commits.
Examples:
* `Fixes: https://github.com/nodejs/node/issues/1337`
* `Refs: https://eslint.org/docs/rules/space-in-parens.html`
* `Refs: https://github.com/nodejs/node/pull/3615`
5. If your commit introduces a breaking change (`semver-major`), it should
contain an explanation about the reason of the breaking change, which
situation would trigger the breaking change, and what is the exact change.
Sample complete commit message:
```text
subsystem: explain the commit in one line
The body of the commit message should be one or more paragraphs, explaining
things in more detail. Please word-wrap to keep columns to 72 characters or
less.
Fixes: https://github.com/nodejs/node/issues/1337
Refs: https://eslint.org/docs/rules/space-in-parens.html
```
If you are new to contributing to Node.js, please try to do your best at
conforming to these guidelines, but do not worry if you get something wrong.
One of the existing contributors will help get things situated and the
contributor landing the pull request will ensure that everything follows
the project guidelines.
### Step 5: Rebase
As a best practice, once you have committed your changes, it is a good idea
to use `git rebase` (not `git merge`) to synchronize your work with the main
repository.
```bash
git fetch upstream HEAD
git rebase FETCH_HEAD
```
This ensures that your working branch has the latest changes from `nodejs/node`.
### Step 6: Test
Bug fixes and features should always come with tests. A
[guide for writing tests in Node.js][] has been
provided to make the process easier. Looking at other tests to see how they
should be structured can also help.
The `test` directory within the `nodejs/node` repository is complex and it is
often not clear where a new test file should go. When in doubt, add new tests
to the `test/parallel/` directory and the right location will be sorted out
later.
Before submitting your changes in a pull request, always run the full Node.js
test suite. To run the tests (including code linting) on Unix / macOS:
```bash
./configure && make -j4 test
```
We can speed up the builds by using [Ninja](https://ninja-build.org/). For more
information, see
[Building Node.js with Ninja](building-node-with-ninja.md).
And on Windows:
```powershell
vcbuild test
```
For some configurations, running all tests might take a long time (an hour or
more). To run a subset of the test suite, see the [running tests][] section of
the Building guide.
### Step 7: Push
Once you are sure your commits are ready to go, with passing tests and linting,
begin the process of opening a pull request by pushing your working branch to
your fork on GitHub.
```bash
git push origin my-branch
```
### Step 8: Opening the pull request
From within GitHub, opening a new pull request will present you with a
[pull request template][]. Please try to do your best at filling out the
details, but feel free to skip parts if you're not sure what to put.
Once opened, pull requests are usually reviewed within a few days.
To get feedback on your proposed change even though it is not ready
to land, use the `Convert to draft` option in the GitHub UI.
Do not use the `wip` label as it might not prevent the PR
from landing before you are ready.
### Step 9: Discuss and update
You will probably get feedback or requests for changes to your pull request.
This is a big part of the submission process so don't be discouraged! Some
contributors may sign off on the pull request right away, others may have
more detailed comments or feedback. This is a necessary part of the process
in order to evaluate whether the changes are correct and necessary.
To make changes to an existing pull request, make the changes to your local
branch, add a new commit with those changes, and push those to your fork.
GitHub will automatically update the pull request.
```bash
git add my/changed/files
git commit
git push origin my-branch
```
If a git conflict arises, it is necessary to synchronize your branch with other
changes that have landed upstream by using `git rebase`:
```bash
git fetch upstream HEAD
git rebase FETCH_HEAD
git push --force-with-lease origin my-branch
```
**Important:** The `git push --force-with-lease` command is one of the few ways
to delete history in `git`. It also complicates the review process, as it won't
allow reviewers to get a quick glance on what changed. Before you use it, make
sure you understand the risks. If in doubt, you can always ask for guidance in
the pull request.
There are a number of more advanced mechanisms for managing commits using
`git rebase` that can be used, but are beyond the scope of this guide.
Feel free to post a comment in the pull request to ping reviewers if you are
awaiting an answer on something. If you encounter words or acronyms that
seem unfamiliar, refer to this
[glossary](https://github.com/nodejs/node/blob/HEAD/glossary.md).
#### Approval and request changes workflow
All pull requests require "sign off" in order to land. Whenever a contributor
reviews a pull request they may find specific details that they would like to
see changed or fixed. These may be as simple as fixing a typo, or may involve
substantive changes to the code you have written. While such requests are
intended to be helpful, they may come across as abrupt or unhelpful, especially
requests to change things that do not include concrete suggestions on _how_ to
change them.
Try not to be discouraged. If you feel that a particular review is unfair,
say so, or contact one of the other contributors in the project and seek their
input. Often such comments are the result of the reviewer having only taken a
short amount of time to review and are not ill-intended. Such issues can often
be resolved with a bit of patience. That said, reviewers should be expected to
be helpful in their feedback, and feedback that is simply vague, dismissive, and
unhelpful is likely safe to ignore.
### Step 10: Landing
In order to land, a pull request needs to be reviewed and [approved][] by
at least two Node.js Collaborators (one collaborator approval is enough if the
pull request has been open for more than 7 days) and pass a
[CI (Continuous Integration) test run][]. After that, as long as there are no
objections from other contributors, the pull request can be merged. If you find
your pull request waiting longer than you expect, see the
[notes about the waiting time](#waiting-until-the-pull-request-gets-landed).
When a collaborator lands your pull request, they will post
a comment to the pull request page mentioning the commit(s) it
landed as. GitHub might show the pull request as `Closed` at this
point, but don't worry. If you look at the branch you raised your
pull request against, you should see a commit with your name on it.
Congratulations and thanks for your contribution!
## Reviewing pull requests
All Node.js contributors who choose to review and provide feedback on Pull
Requests have a responsibility to both the project and the individual making the
contribution. Reviews and feedback must be helpful, insightful, and geared
towards improving the contribution as opposed to simply blocking it. Do not
expect to be able to block a pull request from advancing simply because you say
"No" without giving an explanation. Be open to having your mind changed. Be open
to working with the contributor to make the pull request better.
Reviews that are dismissive or disrespectful of the contributor or any other
reviewers are strictly counter to the [Code of Conduct][].
When reviewing a pull request, the primary goals are for the codebase to improve
and for the person submitting the request to succeed. Even if a pull request
does not land, the submitters should come away from the experience feeling like
their effort was not wasted or unappreciated. Every pull request from a new
contributor is an opportunity to grow the community.
### Review a bit at a time
Do not overwhelm new contributors.
It is tempting to micro-optimize and make everything about relative performance,
perfect grammar, or exact style matches. Do not succumb to that temptation.
Focus first on the most significant aspects of the change:
1. Does this change make sense for Node.js?
2. Does this change make Node.js better, even if only incrementally?
3. Are there clear bugs or larger scale issues that need attending to?
4. Is the commit message readable and correct? If it contains a breaking change
is it clear enough?
When changes are necessary, _request_ them, do not _demand_ them, and do not
assume that the submitter already knows how to add a test or run a benchmark.
Specific performance optimization techniques, coding styles, and conventions
change over time. The first impression you give to a new contributor never does.
Nits (requests for small changes that are not essential) are fine, but try to
avoid stalling the pull request. Most nits can typically be fixed by the
Node.js collaborator landing the pull request but they can also be an
opportunity for the contributor to learn a bit more about the project.
It is always good to clearly indicate nits when you comment: e.g.
`Nit: change foo() to bar(). But this is not blocking.`
If your comments were addressed but were not folded automatically after new
commits or if they proved to be mistaken, please, [hide them][hiding-a-comment]
with the appropriate reason to keep the conversation flow concise and relevant.
### Be aware of the person behind the code
Be aware that _how_ you communicate requests and reviews in your feedback can
have a significant impact on the success of the pull request. Yes, we may land
a particular change that makes Node.js better, but the individual might just
not want to have anything to do with Node.js ever again. The goal is not just
having good code.
### Respect the minimum wait time for comments
There is a minimum waiting time which we try to respect for non-trivial
changes, so that people who may have important input in such a distributed
project are able to respond.
For non-trivial changes, pull requests must be left open for at least 48 hours.
Sometimes changes take far longer to review, or need more specialized review
from subject-matter experts. When in doubt, do not rush.
Trivial changes, typically limited to small formatting changes or fixes to
documentation, may be landed within the minimum 48 hour window.
### Abandoned or stalled pull requests
If a pull request appears to be abandoned or stalled, it is polite to first
check with the contributor to see if they intend to continue the work before
checking if they would mind if you took it over (especially if it just has
nits left). When doing so, it is courteous to give the original contributor
credit for the work they started (either by preserving their name and email
address) in the commit log, or by using an `Author:` meta-data tag in the
commit.
If a pull request has been inactive for more than six months, add the `stalled` label
to it. That will trigger an automation that adds a comment explaining the pull request
may be close for inactivity, giving a heads-up to the contributor before actually
closing it if it remains inactive.
### Approving a change
Any Node.js core collaborator (any GitHub user with commit rights in the
`nodejs/node` repository) is authorized to approve any other contributor's
work. Collaborators are not permitted to approve their own pull requests.
Collaborators indicate that they have reviewed and approve of the changes in
a pull request either by using GitHub's Approval Workflow, which is preferred,
or by leaving an `LGTM` ("Looks Good To Me") comment.
When explicitly using the "Changes requested" component of the GitHub Approval
Workflow, show empathy. That is, do not be rude or abrupt with your feedback
and offer concrete suggestions for improvement, if possible. If you're not
sure _how_ a particular change can be improved, say so.
Most importantly, after leaving such requests, it is courteous to make yourself
available later to check whether your comments have been addressed.
If you see that requested changes have been made, you can clear another
collaborator's `Changes requested` review.
Change requests that are vague, dismissive, or unconstructive may also be
dismissed if requests for greater clarification go unanswered within a
reasonable period of time.
Use `Changes requested` to block a pull request from landing. When doing so,
explain why you believe the pull request should not land along with an
explanation of what may be an acceptable alternative course, if any.
### Accept that there are different opinions about what belongs in Node.js
Opinions on this vary, even among the members of the Technical Steering
Committee.
One general rule of thumb is that if Node.js itself needs it (due to historic
or functional reasons), then it belongs in Node.js. For instance, `url`
parsing is in Node.js because of HTTP protocol support.
Also, functionality that either cannot be implemented outside of core in any
reasonable way, or only with significant pain.
It is not uncommon for contributors to suggest new features they feel would
make Node.js better. These may or may not make sense to add, but as with all
changes, be courteous in how you communicate your stance on these. Comments
that make the contributor feel like they should have "known better" or
ridiculed for even trying run counter to the [Code of Conduct][].
### Performance is not everything
Node.js has always optimized for speed of execution. If a particular change
can be shown to make some part of Node.js faster, it's quite likely to be
accepted. Claims that a particular pull request will make things faster will
almost always be met by requests for performance [benchmark results][] that
demonstrate the improvement.
That said, performance is not the only factor to consider. Node.js also
optimizes in favor of not breaking existing code in the ecosystem, and not
changing working functional code just for the sake of changing.
If a particular pull request introduces a performance or functional
regression, rather than simply rejecting the pull request, take the time to
work _with_ the contributor on improving the change. Offer feedback and
advice on what would make the pull request acceptable, and do not assume that
the contributor should already know how to do that. Be explicit in your
feedback.
### Continuous integration testing
All pull requests that contain changes to code must be run through
continuous integration (CI) testing at [https://ci.nodejs.org/][].
Only Node.js core collaborators and triagers can start a CI testing run. The
specific details of how to do this are included in the new collaborator
[Onboarding guide][]. Usually, a collaborator or triager will start a CI
test run for you as approvals for the pull request come in.
If not, you can ask a collaborator or triager to start a CI run.
Ideally, the code change will pass ("be green") on all platform configurations
supported by Node.js. This means that all tests pass and there are no linting
errors. In reality, however, it is not uncommon for the CI infrastructure itself
to fail on specific platforms or for so-called "flaky" tests to fail ("be red").
It is vital to visually inspect the results of all failed ("red") tests to
determine whether the failure was caused by the changes in the pull request.
## Notes
### Commit squashing
In most cases, do not squash commits that you add to your pull request during
the review process. When the commits in your pull request land, they may be
squashed into one commit per logical change. Metadata will be added to the
commit message (including links to the pull request, links to relevant issues,
and the names of the reviewers). The commit history of your pull request,
however, will stay intact on the pull request page.
For the size of "one logical change",
[0b5191f](https://github.com/nodejs/node/commit/0b5191f15d0f311c804d542b67e2e922d98834f8)
can be a good example. It touches the implementation, the documentation,
and the tests, but is still one logical change. All tests should always pass
when each individual commit lands on one of the `nodejs/node` branches.
### Getting approvals for your pull request
A pull request is approved either by saying LGTM, which stands for
"Looks Good To Me", or by using GitHub's Approve button.
GitHub's pull request review feature can be used during the process.
For more information, check out
[the official documentation](https://help.github.com/articles/reviewing-changes-in-pull-requests/).
After you push new changes to your branch, you need to get
approval for these new changes again, even if GitHub shows "Approved"
because the reviewers have hit the buttons before.
### Waiting until the pull request gets landed
A pull request needs to stay open for at least 48 hours from when it is
submitted, even after it gets approved and passes the CI. This is to make sure
that everyone has a chance to weigh in. If the changes are trivial,
collaborators may decide it doesn't need to wait. A pull request may well take
longer to be merged in. All these precautions are important because Node.js is
widely used, so don't be discouraged!
### Check out the collaborator guide
If you want to know more about the code review and the landing process, see the
[collaborator guide][].
### Appendix: subsystems
* `lib/*.js` (`assert`, `buffer`, etc.)
* `build`
* `doc`
* `lib / src`
* `test`
* `tools`
You can find the full list of supported subsystems in the
[nodejs/core-validate-commit][] repository.
More than one subsystem may be valid for any particular issue or pull request.
[Building guide]: ../../BUILDING.md
[CI (Continuous Integration) test run]: #continuous-integration-testing
[Code of Conduct]: https://github.com/nodejs/admin/blob/HEAD/CODE_OF_CONDUCT.md
[Onboarding guide]: ../../onboarding.md
[approved]: #getting-approvals-for-your-pull-request
[benchmark results]: writing-and-running-benchmarks.md
[collaborator guide]: collaborator-guide.md
[guide for writing tests in Node.js]: writing-tests.md
[hiding-a-comment]: https://help.github.com/articles/managing-disruptive-comments/#hiding-a-comment
[https://ci.nodejs.org/]: https://ci.nodejs.org/
[maintaining dependencies]: ./maintaining/maintaining-dependencies.md
[nodejs/core-validate-commit]: https://github.com/nodejs/core-validate-commit/blob/main/lib/rules/subsystem.js
[pull request template]: https://raw.githubusercontent.com/nodejs/node/HEAD/.github/PULL_REQUEST_TEMPLATE.md
[running tests]: ../../BUILDING.md#running-tests

View File

@ -0,0 +1,77 @@
# Recognizing Contributors
The project looks to recognize and thank contributors for their contributions
as one of the ways we can help them continue to contribute to the project.
This document captures some of the processes we have in place to do that.
Some of the benefits we hope to achieve through these programs include:
* Ensuring people know that the project appreciates their work
* Encouraging people to repeat/grow their contributions
* Helping people demonstrate to their managers the value of the
contributions they are making, which in terms helps their manager support
their employer investing in the project (unfortunately this is
something that matters)
* Highlighting the new features in Node.js and showing that they are
being added by individuals like those mentioned. This should encourage
other people to get involved and make similar contributions.
* Highlighting those leading efforts/initiatives and they are being
done by individuals or teams like those mentioned. This should
encourage other people to get involved and make similar contributions
## Sponsorship link on nodejs/node README.md
Collaborators may add a single link beside their name on the nodejs/node
[`README.md`](../../README.md) file in the section titled `Collaborators`
named `Support me`. This link may be:
1. a direct link to one of the sponsorship platforms listed below,
2. a page that has text explaining how to support the collaborator
including zero or more links to the sponsorship platforms listed below.
The page should not have any outgoing links other than to the
sponsorship platforms. We expect and trust collaborators to ensure that
the text is professional and respectful.
The sponsorship platforms to which links can be made currently
includes (in alphabetical order):
* [GitHub Sponsors](https://github.com/sponsors)
* [Polar](https://polar.sh/)
* [thanks.dev](https://thanks.dev)
## Amplification of posts announcing talks by collaborators
When a collaborator is speaking at a public event they may request amplification
by the Node.js social media accounts. They can request that a post highlighting
that they are speaking at the event be amplified by the Node.js accounts by
making a request in the #nodejs-social channel in the OpenJS Foundation slack.
We trust that collaborators will only request amplification when they believe
the event is relevant to the Node.js community and the content of the post is
in line with the norms of the project.
## Bi-monthly contributor spotlight
The contributor spotlight program showcases individual(s) or teams who have
made signification contributions to the project. Anybody who is not
a member of the Technical Steering Committee (TSC) and has made a
notable contribution to the project can be nominated. They DO NOT need
to be a collaborator, team member etc.
The work of the team or individual(s) will be showcased on the
Node.js social channels and, if warranted, through a blog post created by the
OpenJS marketing staff along with the team or individual(s) which will
be published on [openjsf.org](https://openjsf.org/).
Nominations can be made through this web form:
[Contributor Spotlight nomination](https://forms.gle/dXBXZn8yu83HLJ57A)
Nominations will be reviewed/approved by the Technical Steering Committee (TSC)
on a regular basis as scheduled by the TSC chair. The TSC will also
recommend the order in which approved nominations are processed
by the OpenJS marketing staff.
Before any work proceeds on an approved nomination the individual(s)
or team will be contacted to confirm they want to participate. Social
content and or spotlight blog post will only be published after the
team or individual(s) have confirmed they are happy with the content.

View File

@ -0,0 +1,259 @@
# Node.js Node-API version release process
This document describes the technical aspects of the Node.js Node-API version
release process.
## Table of contents
* [How to create a release](#how-to-create-a-release)
* [0. Pre-release steps](#0-pre-release-steps)
* [1. Update the main branch](#1-update-the-main-branch)
* [2. Create a new branch for the release](#2-create-a-new-branch-for-the-release)
* [3. Update `NODE_API_SUPPORTED_VERSION_MAX`](#3-update-node_api_supported_version_max)
* [4. Define `addon_context_register_func`](#4-define-addon_context_register_func)
* [5. Update version guards](#5-update-version-guards)
* [6. Update version matrix document](#6-update-version-matrix-document)
* [7. Create release commit](#7-create-release-commit)
* [8. Propose release on GitHub](#8-propose-release-on-github)
* [9. Ensure that the release branch is stable](#9-ensure-that-the-release-branch-is-stable)
* [10. Land the release](#10-land-the-release)
* [11. Backport the release](#11-backport-the-release)
## How to create a release
Notes:
* Version strings are listed below as _"vx"_ or _"x"_. Substitute for
the release version.
* Examples will use the integer release version `10`.
### 0. Pre-release steps
Before preparing a Node.js Node-API version release, the Node-API Working Group
must be notified at least one business day in advance of the expected release.
Node-API Working Group can be contacted best by opening up an issue on the
[abi-stable-node issue tracker][].
### 1. Update the main Branch
Checkout the staging branch locally.
```bash
git remote update
git checkout main
git reset --hard upstream/main
```
If the staging branch is not up to date relative to `main`, bring the
appropriate PRs and commits into it.
### 2. Create a new branch for the release
Create a new branch named `node-api-x-proposal`, off the main branch.
```bash
git checkout -b node-api-10-proposal upstream/main
```
### 3. Update `NODE_API_SUPPORTED_VERSION_MAX`
Set the version for the proposed release using the following macros, which are
already defined in `src/node_version.h`:
```c
#define NODE_API_SUPPORTED_VERSION_MAX x
```
> Note: Do not update the `NAPI_VERSION` defined in `src/js_native_api.h`. It
> is a fixed constant baseline version of Node-API.
### 4. Define `addon_context_register_func`
For each new version of Node-API an `else if` case must be added to
`get_node_api_context_register_func` in `src/node_api.cc` and the numeric
literal used in the `static_assert` statement in that function must be updated
to the new Node-API version.
### 5. Update version guards
#### Step 1. Update define version guards
If this release includes new Node-APIs that were first released in this
version, the relevant commits should already include the `NAPI_EXPERIMENTAL`
define guards on the declaration of the new Node-API. Check for these guards
with:
```bash
grep \
-nHE \
'N(ODE_)?API_EXPERIMENTAL' \
src/js_native_api{_types,}.h \
src/node_api{_types,}.h
```
and update the define version guards with the release version:
```diff
- #ifdef NAPI_EXPERIMENTAL
+ #if NAPI_VERSION >= 11
NAPI_EXTERN napi_status NAPI_CDECL
node_api_function(napi_env env);
- #endif // NAPI_EXPERIMENTAL
+ #endif // NAPI_VERSION >= 11
```
Remove any feature flags of the form `NODE_API_EXPERIMENTAL_HAS_<FEATURE>`.
Remove any additional `NODE_API_EXPERIMENTAL_*` guards along with
`NAPI_EXPERIMENTAL`.
Also, update the Node-API version value of the `napi_get_version` test in
`test/js-native-api/test_general/test.js` with the release version `x`:
```diff
// Test version management functions
- assert.strictEqual(test_general.testGetVersion(), 9);
+ assert.strictEqual(test_general.testGetVersion(), 10);
```
#### Step 2. Update runtime version guards
If this release includes runtime behavior version guards, the relevant commits
should already include the `NAPI_VERSION_EXPERIMENTAL` guard for the change.
Check for these guards with:
```bash
grep -nH NAPI_VERSION_EXPERIMENTAL src/js_native_api_v8* src/node_api.cc
```
and substitute this guard version with the release version `x`.
#### Step 3. Update test version guards
If this release includes add-on tests for the new Node-APIs, the relevant
commits should already include `NAPI_EXPERIMENTAL` definition for the tests.
Check for these definitions with:
```bash
grep \
-nHE \
'N(ODE_)?API_EXPERIMENTAL' \
test/node-api/*/{*.{h,c},binding.gyp} \
test/js-native-api/*/{*.{h,c},binding.gyp}
```
and substitute the `NAPI_EXPERIMENTAL` with the release version
`NAPI_VERSION x`;
```diff
- #define NAPI_EXPERIMENTAL
+ #define NAPI_VERSION 10
```
Remove any `NODE_API_EXPERIMENTAL_*` flags.
#### Step 4. Update document
If this release includes new Node-APIs that were first released in this
version and are necessary to document, the relevant commits should already
have documented the new Node-API.
For all Node-API functions and types with define guards updated in Step 1,
in `doc/api/n-api.md`, add the `napiVersion: x` metadata to the Node-API types
and functions that are released in the version, and remove the experimental
stability banner:
```diff
#### node_api_function
<!-- YAML
added:
- v1.2.3
+ napiVersion: 11
-->
- > Stability: 1 - Experimental
```
#### Step 5. Update change history
If this release includes new Node-APIs runtime version guards that were first
released in this version and are necessary to document, the relevant commits
should already have documented the new behavior in a "Change History" section.
For all runtime version guards updated in Step 2, check for these definitions
with:
```bash
grep -nH NAPI_EXPERIMENTAL doc/api/n-api.md
```
In `doc/api/n-api.md`, update the `experimental` change history item to be the
released version `x`:
```diff
Change History:
- * experimental (`NAPI_EXPERIMENTAL` is defined):
+ * version 10:
```
### 6. Update version matrix document
Add a new row in the [version matrix][] for
the new version:
```text
<tr>
<th scope="row">10</th>
<td>vREPLACEME+ and all later versions</td>
</tr>
```
In this case, use `REPLACEME` as a placeholder for the Node.js version to be released.
It will be updated in a Node.js version release.
### 7. Create release commit
When committing these to git, use the following message format:
```text
node-api: define version x
```
### 8. Propose release on GitHub
Create a pull request targeting the `main` branch. These PRs should be left
open for at least 24 hours, and can be updated as new commits land.
If you need any additional information about any of the commits, this PR is a
good place to @-mention the relevant contributors.
Tag the PR with the `notable-change` label, and @-mention the GitHub team
@nodejs/node-api and @nodejs/node-api-implementer.
### 9. Ensure that the release branch is stable
Run a **[`node-test-pull-request`](https://ci.nodejs.org/job/node-test-pull-request/)**
test run to ensure that the build is stable and the HEAD commit is ready for
release.
### 10. Land the release
See the steps documented in [Collaborator Guide - Landing a PR][] to land the
PR.
### 11. Backport the release
Consider backporting the release to all LTS versions following the steps
documented in the [backporting guide][].
Additionally, update the [version matrix][] for the backported version if
necessary.
[Collaborator Guide - Landing a PR]: ./collaborator-guide.md#landing-pull-requests
[abi-stable-node issue tracker]: https://github.com/nodejs/abi-stable-node/issues
[backporting guide]: backporting-to-release-lines.md
[version matrix]: ../api/n-api.md#node-api-version-matrix

1497
doc/contributing/releases.md Normal file

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,62 @@
# Security Model Strategy
A clear security model, with features like permissions and policy enforcement,
is a
[top technical priority](https://github.com/nodejs/node/blob/HEAD/doc/contributing/technical-priorities.md#permissionspoliciessecurity-model)
of Node.js.
## High-level approach
* Document the security model
* Document threat models and current state of the art
* Support experimentation on features like permissions and policies
* Add a security component in Node.js certification covering
the Node.js security model
### Document the security model
The current security model for Node.js is not yet well documented.
At a high level it is:
* Node.js does not provide a sandbox, both the JavaScript and
native code which is run is trusted to not be malicious.
* The project works to help code running on top of Node.js to avoid
making mistakes, but not doing so is not considered a
vulnerability in Node.js. Just because you can build something
vulnerable with the APIs does not mean there is a vulnerability
in Node.js itself.
The project has a goal to better document the security model
and this section will be expanded when that happens.
Once the security model is documented the project will work
to add a security component in Node.js certification covering
the Node.js security model.
### Document threat models and current state of the art
Node.js is used in several different use cases and the
threats may be different in each use case. The project
should document the threat models and use that to
help define the security model in the context of each
of these use cases.
This section will be expanded as the use case/threat
models are defined. The initial list includes:
* Server
* Desktop application
* Cli
* Single executable application
* CI/CD pipeline components
### Support experimentation on features like permissions
The project is not currently planning to provide supported
sandbox functionality, but wants to support experimentation on
related features like permission enforcement.
Features in this category should:
* be opt-in, and additional overhead when not enabled must be low
* limit change in core to just what is needed to enable experimentation

View File

@ -0,0 +1,251 @@
# Security release process
The security release process covers the steps required to plan/implement a
security release. This document is copied into the description of the Next
Security Release and used to track progress on the release. It contains _**TEXT
LIKE THIS**_ which will be replaced during the release process with the
information described.
## Security release stewards
For each security release, a security steward will take ownership for
coordinating the steps outlined in this process. Security stewards
are nominated through an issue in the TSC repository and approved
through the regular TSC consensus process. Once approved, they
are given access to all of the resources needed to carry out the
steps listed in the process as outlined in
[security steward on/off boarding](security-steward-on-off-boarding.md).
The current security stewards are documented in the main Node.js
[README.md](https://github.com/nodejs/node#security-release-stewards).
| Company | Person | Release Date |
| ------------ | --------------- | ------------ |
| NearForm | Matteo | 2021-Oct-12 |
| Datadog | Bryan | 2022-Jan-10 |
| RH and IBM | Joe | 2022-Mar-18 |
| NearForm | Matteo / Rafael | 2022-Jul-07 |
| Datadog | Vladimir | 2022-Sep-23 |
| NodeSource | Juan | 2022-Nov-04 |
| RH and IBM | Michael | 2023-Feb-16 |
| NearForm | Rafael | 2023-Jun-20 |
| NearForm | Rafael | 2023-Aug-09 |
| NearForm | Rafael | 2023-Oct-13 |
| NodeSource | Rafael | 2024-Feb-14 |
| NodeSource | Rafael | 2024-Apr-03 |
| NodeSource | Rafael | 2024-Apr-10 |
| NodeSource | Rafael | 2024-Jul-08 |
| NodeSource | Rafael | 2025-Jan-21 |
| NodeSource | Rafael | 2025-May-14 |
| Datadog | Bryan | |
| IBM | Joe | |
| Platformatic | Matteo | |
| NodeSource | Juan | |
| Red Hat | Michael | |
## Planning
* [ ] 1\. **Generating Next Security Release PR**
* Run `git node security --start` inside [security-release][] repository.
* This command generates a new `vulnerabilities.json` file with HackerOne
reports chosen to be released in the `security-release/next-security-release`
folder.
* It also creates the pull request used to manage the security release.
* [ ] 2\. **Review of Reports:**
* Reports can be added or removed using the following commands:
* Use the "summary" feature in HackerOne. Example [2038134](https://hackerone.com/reports/2038134)
* `git node security --add-report=report_id`
* `git node security --remove-report=report_id`
* Ensure to ping the Node.js TSC team for review of the PRs prior to the release date.
* Adding individuals with expertise in the report topic is also a viable option if
communicated properly with nodejs/security and TSC.
* [ ] 3\. **Assigning Severity and Writing Team Summary:**
* [ ] Assign a severity and write a team summary on HackerOne for the reports
chosen in the `vulnerabilities.json`.
* Run `git node security --sync` to update severity and summary in
`vulnerabilities.json`.
* [ ] 4\. **Requesting CVEs:**
* Request CVEs for the reports with `git node security --request-cve`.
* Make sure to have a green CI before requesting a CVE.
* Check if there is a need to issue a CVE for any version that became
EOL after the last security release through [this issue](https://github.com/nodejs/security-wg/issues/1419).
* [ ] 5\. **Choosing or Updating Release Date:**
* Get agreement on the planned date for the release.
* [ ] Use `git node security --update-date=YYYY/MM/DD` to choose or update the
release date.
* This allows flexibility in postponing the release if needed.
* [ ] 6\. **Get release volunteers:**
* Get volunteers for the upcoming security release on the affected release
lines.
* Make sure to sync nodejs-private (vN.x) branches with nodejs/node.
* [ ] 7\. **Preparing Pre and Post Release Blog Posts:**
* [ ] Create a pre-release blog post using `git node security --pre-release`.
* [ ] Create a post-release blog post using `git node security --post-release`.
## Announcement (one week in advance of the planned release)
* [ ] 1\. **Publish Pre-Release Blog Post:**
* Publish the pre-release blog post on the `nodejs/nodejs.org` repository.
* [ ] 2\. **Send Pre-Release Announcement:**
* Notify the community about the upcoming security release:
* [ ] `git node security --notify-pre-release`
Except for those noted in the list below, this will create automatically the
issues and emails needed for the notifications.
* [docker-node](https://github.com/nodejs/docker-node/issues)
* [build-wg](https://github.com/nodejs/build/issues)
* [ ] (Not yet automatic - do this manually) [Google Groups](https://groups.google.com/g/nodejs-sec)
* Email: notify <oss-security@lists.openwall.com>
* [ ] (Not yet automatic - do this manually) Post in the nodejs-social channel
in the OpenJS slack asking for amplification of the blog post.
```text
Security release pre-alert:
We will release new versions of <add versions> release lines on or shortly
after Day Month Date, Year in order to address:
* # high severity issues
* # moderate severity issues
https://nodejs.org/en/blog/vulnerability/month-year-security-releases/
```
We specifically ask that collaborators other than the releasers and security
steward working on the security release do not tweet or publicize the release
until the tweet from Node.js goes out. We have often
seen tweets sent out before the release is
complete, which may confuse those waiting for the release and take
away from the work the releasers have put into shipping the release.
If the security release will only contain an OpenSSL update, consider
adding the following to the pre-release announcement:
```text
Since this security release will only include updates for OpenSSL, if you're using
a Node.js version which is part of a distribution that uses a system
installed OpenSSL, this Node.js security update may not concern you, instead,
you may need to update your system OpenSSL libraries. Please check the
security announcements for more information.
```
## Release day
* [ ] 1\. **Lock down the CI:**
* Lock down the CI to prevent public access to the CI machines, ping a member of `@nodejs/jenkins-admins`.
* [ ] 2\. **Release:**
* Verify the CI is green on all release proposals (test-V8, CITGM, etc).
* Follow the [release process](https://github.com/nodejs/node/blob/main/doc/contributing/releases.md).
* [ ] 3\. **Unlock the CI:**
* Unlock the CI to allow public access to the CI machines, ping a member of `@nodejs/jenkins-admins`.
* [ ] 4\. **Publish Post-Release Blog Post:**
* Publish the post-release blog post on the `nodejs/nodejs.org` repository.
* [ ] 5\. **Notify the community:**
* Notify the community that the security release has gone out:
* [ ] Slack: `#nodejs-social`
* [ ] [docker-node](https://github.com/nodejs/docker-node/issues)
* [ ] [build-wg](https://github.com/nodejs/build/issues)
* [ ] Email: notify [Google Groups](https://groups.google.com/g/nodejs-sec)
* Forward to <oss-security@lists.openwall.com>
## Post-Release
* [ ] 1\. **Cleanup:**
* [ ] `git node security --cleanup`. This command will:
* Update next-security-release folder
* Close all PRs and backports labeled with `Security Release`.
* Close HackerOne reports:
* Close Resolved
* Request Disclosure
* Request publication of H1 CVE requests
* In case the reporter doesn't accept the disclosure follow this process:
Remove the original report reference within the reference text box and
insert the public URL you would like to be attached to this CVE.
Then uncheck the Public Disclosure on HackerOne box at the bottom of the
page.
![screenshot of HackerOne CVE form](https://github.com/nodejs/node/assets/26234614/e22e4f33-7948-4dd2-952e-2f9166f5568d)
* PR machine-readable JSON descriptions of the vulnerabilities to the [core](https://github.com/nodejs/security-wg/tree/HEAD/vuln/core)
vulnerability DB.
* [ ] Add yourself as a steward in the [Security Release Stewards](https://github.com/nodejs/node/blob/HEAD/doc/contributing/security-release-process.md#security-release-stewards)
## Adding a security revert option
Breaking changes are allowed in existing LTS lines in order to fix
important security vulnerabilities. When breaking changes are made
it is important to provide a command line option that restores
the original behaviour.
The existing Node.js codebase supports the command line
option `--security-revert` and has the boilerplate to make additions
for a specific CVE easy.
To add an option to revert for a CVE, for example `CVE-2024-1234`
simply add this line to
[`node_revert.h`](https://github.com/nodejs/node/blob/main/src/node_revert.h)
```c
XX(CVE_2024_1234, "CVE-2024-1234", "Description of cve")
```
This will allow an easy check of whether a reversion has been
requested or not.
In JavaScript code you can check:
```js
if (process.REVERT_CVE_2024_1234);
```
In C/C++ code you can check:
```c
IsReverted(SECURITY_REVERT_CVE_2024_1234)
```
From the command line a user can request the revert by using
the `--security-revert` option as follows:
```console
node --security-revert=CVE-2024-1234
```
If there are multiple security reverts then multiple instances
of --security-revert can be used. For example:
```console
node --security-revert=CVE-2024-1234 --security-revert=CVE-2024-XXXX
```
## When things go wrong
### Incomplete fixes
When a CVE is reported as fixed in a security release and it turns out that the
fix was incomplete, a new CVE should be used to cover subsequent fix. This
is best practice and avoids confusion that might occur if people believe
they have patched the original CVE by updating their Node.js version and
then we later change the `fixed in` value for the CVE.
### Updating CVEs
The steps to correct CVE information are:
* Go to the “CVE IDs” section in your program
sections (<https://hackerone.com/nodejs/cve_requests>)
* Click the “Request a CVE ID” button
* Enter the CVE ID that needs to be updated
* Include all the details that need updating within the form
* Submit the request
[security-release]: https://github.com/nodejs-private/security-release

View File

@ -0,0 +1,25 @@
# Security Steward Onboarding/OffBoarding
## Onboarding
* Confirm the new steward agrees to keep all private information confidential
to the project and not to use/disclose to their employer.
* Add them to the security-stewards team in the GitHub nodejs-private
organization.
* Add them to the [public website team](https://github.com/orgs/nodejs/teams/website).
* Ensure they have 2FA enabled in H1.
* Add them to the standard team in H1 using this
[page](https://hackerone.com/nodejs/team_members).
* Add them as managers of the
[nodejs-sec](https://groups.google.com/g/nodejs-sec/members) mailing list.
## Offboarding
* Remove them from security-stewards team in the GitHub nodejs-private
organization.
* Remove them from public website team
* Unless they have access for another reason, remove them from the
standard team in H1 using this
[page](https://hackerone.com/nodejs/team_members).
* Downgrade their account to regular member in the
[nodejs-sec](https://groups.google.com/g/nodejs-sec/members) mailing list.

View File

@ -0,0 +1,34 @@
# Sharing project news
The project aims to make it easy
to share project news with those who publish newsletters
that cover Node.js.
The project's approach is as follows:
* The project will create one "What's new in YYYY" discussion item in
the `nodejs/node` repo for news related to Node.js core.
* Teams/groups can create a "what's new in YYYY" issue or discussion item
in one or more of their repositories. As an example see
[What's new in the Node-API and node-addon-api world? - 2023 version](https://github.com/nodejs/abi-stable-node/issues/446).
* Collaborators periodically add news as additional comments on these
issues throughout the year.
* At the end of each year the issue is closed and if appropriate a
new one is created.
* A list of these documents is maintained in the section below as a
quick way to find them.
* The next-10 team reaches out to teams/groups on an ad hoc basis
to ensure teams know about the option for sharing news.
* The next-10 team reaches out to those who publish newsletters ad hoc
to let them know about this approach and how they can find the updates.
We trust/expect collaborators to post project related news versus news
that promotes a specific company or commercial interest.
## Teams doing updates
* [node-api/node-addon-api](https://github.com/nodejs/abi-stable-node/issues/459).
* [uvwasi](https://github.com/nodejs/uvwasi/issues/201).
* [Website team](https://github.com/nodejs/nodejs.org/issues/5602).
* [diagnostics team](https://github.com/nodejs/diagnostics/issues/619).
* [ambassadors](https://github.com/nodejs/nodejs-ambassadors/issues/2).

View File

@ -0,0 +1,19 @@
# Static Analysis
The project uses Coverity to scan Node.js source code and to report potential
issues in the C/C++ code base.
Those who have been added to the [Node.js coverity project][] can receive emails
when there are new issues reported as well as view all current issues
through <https://scan9.scan.coverity.com/reports.htm>.
Any collaborator can ask to be added to the Node.js coverity project
by opening an issue in the [build][] repository titled
`Please add me to coverity`. A member of the build WG with admin access will
verify that the requester is an existing collaborator as listed in the
[collaborators section][] on the nodejs/node project repo. Once validated the
requester will be added to the coverity project.
[Node.js coverity project]: https://scan.coverity.com/projects/node-js
[build]: https://github.com/nodejs/build
[collaborators section]: https://github.com/nodejs/node#collaborators

View File

@ -0,0 +1,49 @@
# Strategic initiatives
The Node.js project has several strategic initiatives underway. A review of the
current initiatives is a standing item on the Technical Steering Committee
agenda to ensure they are active and have the support they need.
## Current initiatives
| Initiative | Champion | Links |
| ---------------------- | -------------------------------- | ------------------------------------------------- |
| Core Promise APIs | [Antoine du Hamel][aduh95] | <https://github.com/nodejs/TSC/issues/1094> |
| QUIC / HTTP3 | [James M Snell][jasnell] | <https://github.com/nodejs/quic> |
| Shadow Realm | [Chengzhong Wu][legendecas] | <https://github.com/nodejs/node/issues/42528> |
| Startup Snapshot | [Joyee Cheung][joyeecheung] | <https://github.com/nodejs/node/issues/35711> |
| V8 Currency | [Michaël Zasso][targos] | |
| Next-10 | [Michael Dawson][mhdawson] | <https://github.com/nodejs/next-10> |
| Single executable apps | [Darshan Sen][RaisinTen] | <https://github.com/nodejs/single-executable> |
| Performance | | <https://github.com/nodejs/performance> |
| Primordials | [Benjamin Gruenbaum][benjamingr] | <https://github.com/nodejs/primordials-use-cases> |
<details>
<summary>List of completed initiatives</summary>
## Completed initiatives
| Initiative | Champion | Links |
| ------------------ | -------------------------- | -------------------------------------------------------------------- |
| Build resources | Michael Dawson | <https://github.com/nodejs/build/issues/1154#issuecomment-448418977> |
| CVE Management | Michael Dawson | <https://github.com/nodejs/security-wg/issues/33> |
| Governance | Myles Borins | |
| Moderation Team | Rich Trott | <https://github.com/nodejs/TSC/issues/329> |
| Modules | Myles Borins | <https://github.com/nodejs/modules> |
| N-API | Michael Dawson | <https://github.com/nodejs/abi-stable-node> |
| npm Integration | Myles Borins | <https://github.com/nodejs/node/pull/21594> |
| OpenSSL Evolution | Rod Vagg | <https://github.com/nodejs/TSC/issues/677> |
| Open Web Standards | Myles Borins, Joyee Cheung | <https://github.com/nodejs/open-standards> |
| VM module fix | Franziska Hinkelmann | <https://github.com/nodejs/node/issues/6283> |
| Workers | Anna Henningsen | <https://github.com/nodejs/worker> |
</details>
[RaisinTen]: https://github.com/RaisinTen
[aduh95]: https://github.com/aduh95
[benjamingr]: https://github.com/benjamingr
[jasnell]: https://github.com/jasnell
[joyeecheung]: https://github.com/joyeecheung
[legendecas]: https://github.com/legendecas
[mhdawson]: https://github.com/mhdawson
[targos]: https://github.com/targos

View File

@ -0,0 +1,117 @@
# Streaming Meetings to Youtube
We publicly live stream our meetings to YouTube using [Zoom](https://zoom.us/).
## Getting Access
You need the Foundation login credentials for Zoom in order to host the meeting.
The userid and password is shared through 1password.
Your YouTube Account must be a manager of the
[Node.js YouTube account](https://www.youtube.com/channel/UCQPYJluYC_sn_Qz_XE-YbTQ).
To request access open an issue in the Node.js
[admin](https://github.com/nodejs/admin) repository with the title being
`Zoom and Youtube access for X` where X is the GitHub id and
the YouTube ID of the person for which access is being requested.
Include a short reason why access is needed(for example streaming
a team meeting etc.).
Unless there are objections the request is considered approved after 48 hours.
## Managing access
### Youtube
To add managers or verify an account is a manager:
1. Go to <https://youtube.com>
2. Click on the Node.js icon on the right top.
3. Select settings, select "Add or remove managers", select "Manage permissions"
4. On that page you can use the +people at the top right of the popup to add
people. It also lists all current managers.
### Zoom
To share the Zoom password log into 1password, select the settings gear for
the`zoom-creds` vault and then use `Share Vault` to share the vault with the
new user. Use the gear to set the permissions to include only `View Items` and
`View and Copy passwords`.
When adding access for a user also ask them to create a PR adding themselves
to the `zoom-nodejs` group in the
[iojs.org/aliases.json](https://github.com/nodejs/email/blob/main/iojs.org/aliases.json)
file [nodejs/email](https://github.com/nodejs/email/)
## Live streaming a meeting
### Start and Stop the stream
1. Login to <https://zoom.us> using the Foundation credentials.
2. Go to <https://zoom.us/webinar/list>, find the meeting.
3. Press "Start", it should open the meeting in the Zoom application.
4. Go to "Participants" panel, check Attendees, promote them to panelists.
5. Go to "... More" in toolbar, choose "Live on YouTube", it will open in
browser.
6. Choose to login to <https://youtube.com> with Node.js account, accept
Zoom usage agreement (on first use)
7. On the Streaming page, edit the webinar title to include the meeting date,
then press the red "Go Live!" button. Troubleshooting note: at least one
person has found that "Go Live!" errored with a message "Please grant
necessary privilege for live streaming". Copying the link from the default
browser to a different browser may work around this issue.
Every participant can choose whether to participate with or without video.
YouTube records the live stream. Recordings are made available on the
[Node.js channel](https://www.youtube.com/channel/UCQPYJluYC_sn_Qz_XE-YbTQ/videos).
The stream title is set automatically from the information in Zoom. We usually
set it to `YYYY-MM-DD - Meeting Name` for example
`2022-11-02 - Technical Steering Community Meeting`.
The description should be a link to the meeting issue.
You can edit title and description on YouTube at a later time if needed.
![YouTube Basic Info example text](./doc_img/youtube-stream-title-description.png)
### Share the meeting once you've gone live
The meeting link should be `http://www.youtube.com/c/nodejs-foundation/live`.
Send it in a tweet such as:
```text
.@nodejs Technical Steering Committee meeting live now:
http://www.youtube.com/c/nodejs-foundation/live
```
Adjust the `Technical Steering Committee` part as necessary and remove the
`.@nodejs` if tweeting from the official twitter account.
![YouTube Share input box](./doc_img/youtube-stream-share.png)
## Check stream status
This should say online when you are streaming, and usually be green.
However, it may turn yellow and issue a warning in a "stream health"
section below. Since we usually stream with static images for the
video, there will often be warnings that the video bitrate is low.
This is a not a problem and should almost always be ignored.
![YouTube Stream Status showing Offline](./doc_img/youtube-stream-status.png)
## Check how many people are watching
![YouTube Analytics graph](./doc_img/youtube-stream-analytics.png)
## Moderate the chat and solicit questions
Moderation follows the [Moderation Policy](https://github.com/nodejs/admin/blob/main/Moderation-Policy.md).
Messages can be moderated right-clicking and selecting the necessary action,
such as `remove`.
If you participate in the chat while logged in as Node.js, it's good
practice to append your initials to your messages.

View File

@ -0,0 +1,5 @@
# Suggesting Social Media Posts
Node.js social media is managed by OpenJS Foundation staff. The processes are
documented in
[Node.js Social Amplification Request Guidelines](https://docs.google.com/document/d/1yrYZJ2twrbpUuScbo3rmN_v-Jfv6d2tO74nCT6PcpxI).

View File

@ -0,0 +1,177 @@
# Technical Priorities
This list represents the current view of key technical priorities recognized
by the project as important to ensure the ongoing success of Node.js.
It is based on an understanding of the Node.js
[constituencies](https://github.com/nodejs/next-10/blob/main/CONSTITUENCIES.md)
and their [needs](https://github.com/nodejs/next-10/blob/main/CONSTITUENCY-NEEDS.md).
The initial version was created based on the work of the
[Next-10 team](https://github.com/nodejs/next-10) and the
[mini-summit](https://github.com/nodejs/next-10/issues/76)
on August 5th 2021.
They will be updated regularly and will be reviewed by the next-10 team
and the TSC on a 6-month basis.
Version from the [mini-summit](https://github.com/nodejs/next-10/issues/1)
on October 1st 2022.
## Modern HTTP
_Present in: 2021_
Base HTTP support is a key component of modern cloud-native applications,
and built-in support was part of what made Node.js a success in the first
10 years. The current implementation is hard to support and a common
source of vulnerabilities. We must work towards an
implementation that is easier to support and makes it easier to integrate
the new HTTP versions (HTTP3, QUIC) and to support efficient
implementations of different versions concurrently.
## Suitable types for end-users
_Present in: 2021_
Using typings with JavaScript can allow a richer experience when using Visual
Studio Code (or any other IDEs) environments, more complete documentation
of APIs and the ability to identify and resolve errors earlier in the
development process. These benefits are important to a large number of Node.js
developers (maybe 50%). Further typing support may be important
to enterprises that are considering expanding their preferred platforms to
include Node.js. It is, therefore, important that the Node.js project work
to ensure there are good typings available for the public Node.js APIs.
## Documentation
_Present in: 2021_
The current documentation is great for experienced developers or people
who are aware of what they are looking for. On the other hand, for
beginners this documentation can be quite hard to read and finding the
desired information is difficult. We must have documentation
that is suitable for beginners to continue the rapid growth in use.
This documentation should include more concrete examples and a learning
path for newcomers.
## WebAssembly
_Present in: 2021_
The use of WebAssembly has been growing over the last few years.
To ensure Node.js continues to be part of solutions where a
subset of the solution needs the performance that WebAssembly can
deliver, Node.js must provide good support for running
WebAssembly components along with the JavaScript that makes up the rest
of the solution. This includes implementations of “host” APIs like WASI.
## ES Modules (ESM)
_Present in: 2021_
The CommonJS module system was one of the key components that led to the
success of Node.js in its first 10 years. ESM is the standard that has
been adopted as the equivalent in the broader JavaScript ecosystem and
Node.js must continue to develop and improve its ESM implementation
to stay relevant and ensure continued growth for the next 10 years.
## Support for features from the latest ECMAScript spec
JavaScript developers are a fast moving group and need/want support for new ES
JavaScript features in a timely manner. Node.js must continue
to provide support for up to date ES versions to remain the runtime
of choice and to ensure its continued growth for the next 10 years.
## Observability
_Present in: 2021_
The ability to investigate and resolve problems that occur in applications
running in production is crucial for organizations. Tools that allow
people to observe the current and past operation of the application are
needed to support that need. It is therefore important that the Node.js
project work towards well understood and defined processes for observing
the behavior of Node.js applications as well as ensuring there are well
supported tools to implement those processes (logging, metrics and tracing).
This includes support within the Node.js runtime itself (for example
generating heap dumps, performance metrics, etc.) as well as support for
applications on top of the runtime. In addition, it is also important to
clearly document the use cases, problem determination methods, and best
practices for those tools.
## Better multithreaded support
_Present in: 2021_
Today's servers support multiple threads of concurrent execution.
Node.js deployments must be able to make full and efficient
use of the available resources. The right answer is often to use
technologies like containers to run multiple single threaded Node.js
instances on the same server. However, there are important use cases
where a single Node.js instance needs to make use of multiple threads
to achieve a performant and efficient implementation. In addition,
even when a Node.js instance only needs to consume a single thread to
complete its work there can be issues. If that work is long running,
blocking the event loop will interfere with other supporting work like
metrics gathering and health checks. Node.js
must provide good support for using multiple threads
to ensure the continued growth and success of Node.js.
## Single Executable Applications
Node.js often loses out to other runtimes/languages in cases where
being able to package a single, executable application simplifies
distribution and management of what needs to be delivered. While there are
components/approaches for doing this, they need to be better
documented and evangelized so that this is not seen as a barrier
for using Node.js in these situations. This is important to support
the expansion of where/when Node.js is used in building solutions.
## Serverless
Serverless is a cloud computing model where the cloud provider manages the
underlying infrastructure and automatically allocates resources as
needed. Developers only need to focus on writing code for specific
functions, which are executed as individual units of work in response to
events. Node.js is one of the main technology used by developers in
this field therefore it is crucial for us to provide a great solution.
## Small footprint
Small software footprints refer to software that has a minimal impact on
system resources such as memory and processing power. This can be achieved
through various methods such as optimizing code, reducing the number of
dependencies, or using lightweight frameworks. Smaller footprints can lead
to faster startup times, reduced memory usage, and improved overall system
performance. This is fundamental for Node.js to be a lightweight proposition
inside the ecosystem as it is used across a wild variety of projects, from
web application to IoT and serverless.
## Developers-first DX
Developer experience (DX) refers to the overall experience a developer has when
working with a software development platform, framework, or tool. It encompasses
all aspects of the developer's interactions with the system, from installation
and configuration to writing code and debugging. A good DX prioritizes ease
of use, efficiency, and productivity, and can lead to faster development times,
higher quality code, and greater developer satisfaction. Factors that can
impact DX include documentation, community support, testing tools,
and integration with other systems.
As TypeScript usage continues to grow and gains more prominence in the
ecosystem, enhancing its support is essential for delivering an improved
developer experience for newcomers and experienced users alike.
## Package management
The ability to easily install and manage dependencies and development tools is a
key part of the user experience, and for that reason Node.js must provide a
package manager as part of its distribution. Node.js includes `npm` for this
purpose. This is for historical reasons — when `npm` was added in 2011, it was
the only JavaScript package manager — and because it is the reference
implementation for the npm registry, which is the de facto primary source for
most JavaScript software. In accordance with our [policy][distribution-policy]
of not including multiple dependencies or tools that serve the same purpose, the
Node.js project does not include any other package managers; though it may
include other software to download other package managers.
[distribution-policy]: ./distribution.md

View File

@ -0,0 +1,84 @@
# Technical values and their priorities
The project uses these technical values to establish priorities and guide
collaboration.
These are the shared values as of this writing and will
evolve. We hope they are useful to people new
to the project in order to better understand which contributions
will be aligned with the current direction and as thinking
points when trading off between conflicting goals.
The factors influencing every discussion/decision are
different and priority 1 does not always trump priority 2
and so on.
## Values and priority level
* Priority 1 - Developer experience
* Priority 2 - Stability
* Priority 3 - Operational qualities
* Priority 4 - Node.js maintainer experience
* Priority 5 - Up to date technology and APIs
## Value descriptions
### 1 - Developer experience
We value ensuring that developers are productive and enjoy developing
with Node.js. Some key elements of this include:
* Approachability (both technical and community)
* Great documentation
* Bundling friction-reducing APIs and components, even though
they could be provided externally
* Compatibility and interoperability with browsers and other JavaScript
environments so that as much code as possible runs as is both in Node.js and
in the other environments
* Enabling/supporting external packages to ensure overall developer experience
### 2 - Stability
Whenever possible, we seek to ensure that working code continues to work. To
keep the trust of developers and users, we value stability.
Some key elements of this include:
* Backward compatibility
* Stable releases on a predictable schedule
* A strong safety net, including testing how changes
in Node.js affect popular packages
* Careful consideration of what goes into Long Term Support (LTS) releases
### 3 - Operational qualities
We value keeping Node.js safe, performant, and lightweight.
We value enabling the ability to investigate and debug problems in
development and production. Some key elements of this include:
* High throughput (speed)
* Fast startup
* Small binary size
* Small memory footprint
* High-quality debugging tools
* Robust diagnostic tools (profilers, etc.)
* Responsible security practices
### 4 - Node.js maintainer experience
We value the productivity and happiness of the Node.js maintainers.
Some key elements of this include:
* Approachability of the codebase
* Good internal documentation and guides
* Low-friction policies and processes
* Good CI and tooling to make maintainers productive
### 5 - Up to date technology and APIs
We value providing developers with modern APIs and technologies
following existing standards whenever possible.
Some key elements of this include:
* Participating in standards work and organizations
* Web API compatibility
* Supporting and exposing new technologies and standards through early adoption

View File

@ -0,0 +1,149 @@
# Using the internal/errors.js module
## What is internal/errors.js
The `require('internal/errors')` module is an internal-only module that can be
used to produce `Error`, `TypeError` and `RangeError` instances that use a
static, permanent error code and an optionally parameterized message.
The intent of the module is to allow errors provided by Node.js to be assigned a
permanent identifier. Without a permanent identifier, userland code may need to
inspect error messages to distinguish one error from another. An unfortunate
result of that practice is that changes to error messages result in broken code
in the ecosystem. For that reason, Node.js has considered error message changes
to be breaking changes. By providing a permanent identifier for a specific
error, we reduce the need for userland code to inspect error messages.
Switching an existing error to use the `internal/errors` module must be
considered a `semver-major` change.
## Using internal/errors.js
The `internal/errors` module exposes all custom errors as subclasses of the
builtin errors. After being added, an error can be found in the `codes` object.
For instance, an existing `Error` such as:
```js
const err = new TypeError(`Expected string received ${type}`);
```
Can be replaced by first adding a new error key into the `internal/errors.js`
file:
```js
E('FOO', 'Expected string received %s', TypeError);
```
Then replacing the existing `new TypeError` in the code:
```js
const { FOO } = require('internal/errors').codes;
// ...
const err = new FOO(type);
```
## Adding new errors
New static error codes are added by modifying the `internal/errors.js` file
and appending the new error codes to the end using the utility `E()` method.
```js
E('EXAMPLE_KEY1', 'This is the error value', TypeError);
E('EXAMPLE_KEY2', (a, b) => `${a} ${b}`, RangeError);
```
The first argument passed to `E()` is the static identifier. The second
argument is either a String with optional `util.format()` style replacement
tags (e.g. `%s`, `%d`), or a function returning a String. The optional
additional arguments passed to the `errors.message()` function (which is
used by the `errors.Error`, `errors.TypeError` and `errors.RangeError` classes),
will be used to format the error message. The third argument is the base class
that the new error will extend.
It is possible to create multiple derived
classes by providing additional arguments. The other ones will be exposed as
properties of the main class:
<!-- eslint-disable no-unreachable -->
```js
E('EXAMPLE_KEY', 'Error message', TypeError, RangeError);
// In another module
const { EXAMPLE_KEY } = require('internal/errors').codes;
// TypeError
throw new EXAMPLE_KEY();
// RangeError
throw new EXAMPLE_KEY.RangeError();
```
## Documenting new errors
Whenever a new static error code is added and used, corresponding documentation
for the error code should be added to the `doc/api/errors.md` file. This will
give users a place to go to easily look up the meaning of individual error
codes.
In case `make lint` fails to detect the new error codes added into `errors.md`,
the markdown linting cache must be cleaned with `make lint-md-clean`.
## Testing new errors
When adding a new error, corresponding test(s) for the error message
formatting may also be required. If the message for the error is a
constant string then no test is required for the error message formatting
as we can trust the error helper implementation. An example of this kind of
error would be:
```js
E('ERR_SOCKET_ALREADY_BOUND', 'Socket is already bound');
```
If the error message is not a constant string then tests to validate
the formatting of the message based on the parameters used when
creating the error should be added to
`test/parallel/test-internal-errors.js`. These tests should validate
all of the different ways parameters can be used to generate the final
message string. A simple example is:
```js
// Test ERR_TLS_CERT_ALTNAME_INVALID
assert.strictEqual(
errors.message('ERR_TLS_CERT_ALTNAME_INVALID', ['altname']),
'Hostname/IP does not match certificate\'s altnames: altname');
```
In addition, there should also be tests which validate the use of the
error based on where it is used in the codebase. If the error message is
static, these tests should only validate that the expected code is received
and NOT validate the message. This will reduce the amount of test change
required when the message for an error changes.
```js
assert.throws(() => {
socket.bind();
}, common.expectsError({
code: 'ERR_SOCKET_ALREADY_BOUND',
type: Error,
}));
```
Avoid changing the format of the message after the error has been created.
If it does make sense to do this for some reason, then additional tests
validating the formatting of the error message for those cases will
likely be required.
## API
### Object: errors.codes
Exposes all internal error classes to be used by Node.js APIs.
### Method: errors.message(key, args)
* `key` {string} The static error identifier
* `args` {Array} Zero or more optional arguments passed as an Array
* Returns: {string}
Returns the formatted error message string for the given `key`.

View File

@ -0,0 +1,71 @@
# Using global symbols
ES6 introduced a new type: `Symbol`. This new type is _immutable_, and
it is often used for metaprogramming purposes, as it can be used as
property keys like string. There are two types of
symbols, local and global.
Symbol-keyed properties of an object are not included in the output of
`JSON.stringify()`, but the `util.inspect()` function includes them by
default.
Learn more about symbols at
<https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Symbol>.
## `Symbol(string)`
Symbols created via `Symbol(string)` are local to the caller function.
For this reason, we often use them to simulate private fields, like so:
```js
const kField = Symbol('kField');
console.log(kField === Symbol('kField')); // false
class MyObject {
constructor() {
this[kField] = 'something';
}
}
module.exports.MyObject = MyObject;
```
Symbols are not fully private, as the data could be accessed anyway:
```js
for (const s of Object.getOwnPropertySymbols(obj)) {
const desc = s.toString().replace(/Symbol\((.*)\)$/, '$1');
if (desc === 'kField') {
console.log(obj[s]); // 'something'
}
}
```
Local symbols make it harder for developers to monkey patch/access
private fields, as they require more work than a property prefixed
with an `_`. Monkey patching private API that were not designed to be
monkey-patchable make maintaining and evolving Node.js harder, as private
properties are not documented and can change within a patch release.
Some extremely popular modules in the ecosystem monkey patch some
internals, making it impossible for us to update and improve those
areas without causing issues for a significant amount of users.
## `Symbol.for(string)`
Symbols created with `Symbol.for(string)` are global and unique to the
same V8 Isolate. On the first call to `Symbol.for(string)` a symbol is
stored in a global registry and easily retrieved for every call of
`Symbol.for(string)`. However, this might cause problems when two module
authors use the same symbol
for different reasons.
```js
const s = Symbol.for('hello');
console.log(s === Symbol.for('hello')); // true
```
In the Node.js runtime we prefix all our global symbols with `nodejs.`,
e.g. `Symbol.for('nodejs.hello')`.
Global symbols should be preferred when a developer-facing interface is
needed to allow behavior customization, i.e., metaprogramming.

View File

@ -0,0 +1,667 @@
# How to write and run benchmarks in Node.js core
## Table of contents
* [Prerequisites](#prerequisites)
* [HTTP benchmark requirements](#http-benchmark-requirements)
* [HTTPS benchmark requirements](#https-benchmark-requirements)
* [HTTP/2 benchmark requirements](#http2-benchmark-requirements)
* [Benchmark analysis requirements](#benchmark-analysis-requirements)
* [Running benchmarks](#running-benchmarks)
* [Running individual benchmarks](#running-individual-benchmarks)
* [Running all benchmarks](#running-all-benchmarks)
* [Specifying CPU Cores for Benchmarks with run.js](#specifying-cpu-cores-for-benchmarks-with-runjs)
* [Filtering benchmarks](#filtering-benchmarks)
* [Comparing Node.js versions](#comparing-nodejs-versions)
* [Comparing parameters](#comparing-parameters)
* [Running benchmarks on the CI](#running-benchmarks-on-the-ci)
* [Creating a benchmark](#creating-a-benchmark)
* [Basics of a benchmark](#basics-of-a-benchmark)
* [Creating an HTTP benchmark](#creating-an-http-benchmark)
## Prerequisites
Basic Unix tools are required for some benchmarks.
[Git for Windows][git-for-windows] includes Git Bash and the necessary tools,
which need to be included in the global Windows `PATH`.
### HTTP benchmark requirements
Most of the HTTP benchmarks require a benchmarker to be installed. This can be
either [`wrk`][wrk] or [`autocannon`][autocannon].
`Autocannon` is a Node.js script that can be installed using
`npm install -g autocannon`. It will use the Node.js executable that is in the
path. In order to compare two HTTP benchmark runs, make sure that the
Node.js version in the path is not altered.
`wrk` may be available through one of the available package managers. If not,
it can be easily built [from source][wrk] via `make`.
By default, `wrk` will be used as the benchmarker. If it is not available,
`autocannon` will be used in its place. When creating an HTTP benchmark, the
benchmarker to be used should be specified by providing it as an argument:
`node benchmark/run.js --set benchmarker=autocannon http`
`node benchmark/http/simple.js benchmarker=autocannon`
#### HTTPS benchmark requirements
To run the `https` benchmarks, one of `autocannon` or `wrk` benchmarkers must
be used.
`node benchmark/https/simple.js benchmarker=autocannon`
#### HTTP/2 benchmark requirements
To run the `http2` benchmarks, the `h2load` benchmarker must be used. The
`h2load` tool is a component of the `nghttp2` project and may be installed
from [nghttp2.org][] or built from source.
`node benchmark/http2/simple.js benchmarker=h2load`
### Benchmark analysis requirements
To analyze the results statistically, you can use either the
[node-benchmark-compare][] tool or the R script `benchmark/compare.R`.
[node-benchmark-compare][] is a Node.js script that can be installed with
`npm install -g node-benchmark-compare`.
To draw comparison plots when analyzing the results, `R` must be installed.
Use one of the available package managers or download it from
<https://www.r-project.org/>.
The R packages `ggplot2` and `plyr` are also used and can be installed using
the R REPL.
```console
$ R
install.packages("ggplot2")
install.packages("plyr")
```
If a message states that a CRAN mirror must be selected first, specify a mirror
with the `repo` parameter.
```r
install.packages("ggplot2", repo="http://cran.us.r-project.org")
```
Of course, use an appropriate mirror based on location.
A list of mirrors is [located here](https://cran.r-project.org/mirrors.html).
## Running benchmarks
### Setting CPU Frequency scaling governor to "performance"
It is recommended to set the CPU frequency to `performance` before running
benchmarks. This increases the likelihood of each benchmark achieving peak performance
according to the hardware. Therefore, run:
```console
$ ./benchmark/cpu.sh fast
```
### Running individual benchmarks
This can be useful for debugging a benchmark or doing a quick performance
measure. But it does not provide the statistical information to make any
conclusions about the performance.
Individual benchmarks can be executed by simply executing the benchmark script
with node.
```console
$ node benchmark/buffers/buffer-tostring.js
buffers/buffer-tostring.js n=10000000 len=0 arg=true: 62710590.393305704
buffers/buffer-tostring.js n=10000000 len=1 arg=true: 9178624.591787899
buffers/buffer-tostring.js n=10000000 len=64 arg=true: 7658962.8891432695
buffers/buffer-tostring.js n=10000000 len=1024 arg=true: 4136904.4060201733
buffers/buffer-tostring.js n=10000000 len=0 arg=false: 22974354.231509723
buffers/buffer-tostring.js n=10000000 len=1 arg=false: 11485945.656765845
buffers/buffer-tostring.js n=10000000 len=64 arg=false: 8718280.70650129
buffers/buffer-tostring.js n=10000000 len=1024 arg=false: 4103857.0726124765
```
Each line represents a single benchmark with parameters specified as
`${variable}=${value}`. Each configuration combination is executed in a separate
process. This ensures that benchmark results aren't affected by the execution
order due to V8 optimizations. **The last number is the rate of operations
measured in ops/sec (higher is better).**
Furthermore a subset of the configurations can be specified, by setting them in
the process arguments:
```console
$ node benchmark/buffers/buffer-tostring.js len=1024
buffers/buffer-tostring.js n=10000000 len=1024 arg=true: 3498295.68561504
buffers/buffer-tostring.js n=10000000 len=1024 arg=false: 3783071.1678948295
```
### Running all benchmarks
Similar to running individual benchmarks, a group of benchmarks can be executed
by using the `run.js` tool. To see how to use this script,
run `node benchmark/run.js`. Again this does not provide the statistical
information to make any conclusions.
```console
$ node benchmark/run.js assert
assert/deepequal-buffer.js
assert/deepequal-buffer.js method="deepEqual" strict=0 len=100 n=20000: 773,200.4995493788
assert/deepequal-buffer.js method="notDeepEqual" strict=0 len=100 n=20000: 964,411.712953848
...
assert/deepequal-map.js
assert/deepequal-map.js method="deepEqual_primitiveOnly" strict=0 len=500 n=500: 20,445.06368453332
assert/deepequal-map.js method="deepEqual_objectOnly" strict=0 len=500 n=500: 1,393.3481642240833
...
assert/deepequal-object.js
assert/deepequal-object.js method="deepEqual" strict=0 size=100 n=5000: 1,053.1950937538475
assert/deepequal-object.js method="notDeepEqual" strict=0 size=100 n=5000: 9,734.193251965213
...
```
It is possible to execute more groups by adding extra process arguments.
```bash
node benchmark/run.js assert async_hooks
```
It's also possible to execute the benchmark more than once using the
`--runs` flag.
```bash
node benchmark/run.js --runs 10 assert async_hooks
```
This command will run the benchmark files in `benchmark/assert` and `benchmark/async_hooks`
10 times each.
#### Specifying CPU Cores for Benchmarks with run.js
When using `run.js` to execute a group of benchmarks,
you can specify on which CPU cores the
benchmarks should execute
by using the `--set CPUSET=value` option.
This controls the CPU core
affinity for the benchmark process,
potentially reducing
interference from other processes and allowing
for performance
testing under specific hardware configurations.
The `CPUSET` option utilizes the `taskset` command's format
for setting CPU affinity, where `value` can be a single core
number or a range of cores.
Examples:
* `node benchmark/run.js --set CPUSET=0` ... runs benchmarks on CPU core 0.
* `node benchmark/run.js --set CPUSET=0-2` ...
specifies that benchmarks should run on CPU cores 0 to 2.
Note: This option is only applicable when using `run.js`.
Ensure the `taskset` command is available on your system
and the specified `CPUSET` format matches its requirements.
#### Filtering benchmarks
`benchmark/run.js` and `benchmark/compare.js` have `--filter pattern` and
`--exclude pattern` options, which can be used to run a subset of benchmarks or
to exclude specific benchmarks from the execution, respectively.
```console
$ node benchmark/run.js --filter "deepequal-b" assert
assert/deepequal-buffer.js
assert/deepequal-buffer.js method="deepEqual" strict=0 len=100 n=20000: 773,200.4995493788
assert/deepequal-buffer.js method="notDeepEqual" strict=0 len=100 n=20000: 964,411.712953848
$ node benchmark/run.js --exclude "deepequal-b" assert
assert/deepequal-map.js
assert/deepequal-map.js method="deepEqual_primitiveOnly" strict=0 len=500 n=500: 20,445.06368453332
assert/deepequal-map.js method="deepEqual_objectOnly" strict=0 len=500 n=500: 1,393.3481642240833
...
assert/deepequal-object.js
assert/deepequal-object.js method="deepEqual" strict=0 size=100 n=5000: 1,053.1950937538475
assert/deepequal-object.js method="notDeepEqual" strict=0 size=100 n=5000: 9,734.193251965213
...
```
`--filter` and `--exclude` can be repeated to provide multiple patterns.
```console
$ node benchmark/run.js --filter "deepequal-b" --filter "deepequal-m" assert
assert/deepequal-buffer.js
assert/deepequal-buffer.js method="deepEqual" strict=0 len=100 n=20000: 773,200.4995493788
assert/deepequal-buffer.js method="notDeepEqual" strict=0 len=100 n=20000: 964,411.712953848
assert/deepequal-map.js
assert/deepequal-map.js method="deepEqual_primitiveOnly" strict=0 len=500 n=500: 20,445.06368453332
assert/deepequal-map.js method="deepEqual_objectOnly" strict=0 len=500 n=500: 1,393.3481642240833
$ node benchmark/run.js --exclude "deepequal-b" --exclude "deepequal-m" assert
assert/deepequal-object.js
assert/deepequal-object.js method="deepEqual" strict=0 size=100 n=5000: 1,053.1950937538475
assert/deepequal-object.js method="notDeepEqual" strict=0 size=100 n=5000: 9,734.193251965213
...
assert/deepequal-prims-and-objs-big-array-set.js
assert/deepequal-prims-and-objs-big-array-set.js method="deepEqual_Array" strict=0 len=20000 n=25 primitive="string": 865.2977195251661
assert/deepequal-prims-and-objs-big-array-set.js method="notDeepEqual_Array" strict=0 len=20000 n=25 primitive="string": 827.8297281403861
assert/deepequal-prims-and-objs-big-array-set.js method="deepEqual_Set" strict=0 len=20000 n=25 primitive="string": 28,826.618268696366
...
```
If `--filter` and `--exclude` are used together, `--filter` is applied first,
and `--exclude` is applied on the result of `--filter`:
```console
$ node benchmark/run.js --filter "bench-" process
process/bench-env.js
process/bench-env.js operation="get" n=1000000: 2,356,946.0770617095
process/bench-env.js operation="set" n=1000000: 1,295,176.3266261867
process/bench-env.js operation="enumerate" n=1000000: 24,592.32231990992
process/bench-env.js operation="query" n=1000000: 3,625,787.2150573144
process/bench-env.js operation="delete" n=1000000: 1,521,131.5742806569
process/bench-hrtime.js
process/bench-hrtime.js type="raw" n=1000000: 13,178,002.113936031
process/bench-hrtime.js type="diff" n=1000000: 11,585,435.712423025
process/bench-hrtime.js type="bigint" n=1000000: 13,342,884.703919787
$ node benchmark/run.js --filter "bench-" --exclude "hrtime" process
process/bench-env.js
process/bench-env.js operation="get" n=1000000: 2,356,946.0770617095
process/bench-env.js operation="set" n=1000000: 1,295,176.3266261867
process/bench-env.js operation="enumerate" n=1000000: 24,592.32231990992
process/bench-env.js operation="query" n=1000000: 3,625,787.2150573144
process/bench-env.js operation="delete" n=1000000: 1,521,131.5742806569
```
#### Grouping benchmarks
Benchmarks can also have groups, giving the developer greater flexibility in differentiating between test cases
and also helping reduce the time to run the combination of benchmark parameters.
By default, all groups are executed when running the benchmark.
However, it is possible to specify individual groups by setting the
`NODE_RUN_BENCHMARK_GROUPS` environment variable when running `compare.js`:
```bash
NODE_RUN_BENCHMARK_GROUPS=fewHeaders,manyHeaders node http/headers.js
```
### Comparing Node.js versions
To compare the effect of a new Node.js version use the `compare.js` tool. This
will run each benchmark multiple times, making it possible to calculate
statistics on the performance measures. To see how to use this script,
run `node benchmark/compare.js`.
As an example on how to check for a possible performance improvement, the
[#5134](https://github.com/nodejs/node/pull/5134) pull request will be used as
an example. This pull request _claims_ to improve the performance of the
`node:string_decoder` module.
First build two versions of Node.js, one from the `main` branch (here called
`./node-main`) and another with the pull request applied (here called
`./node-pr-5134`).
To run multiple compiled versions in parallel you need to copy the output of the
build: `cp ./out/Release/node ./node-main`. Check out the following example:
```bash
git checkout main
./configure && make -j4
cp ./out/Release/node ./node-main
git checkout pr-5134
./configure && make -j4
cp ./out/Release/node ./node-pr-5134
```
The `compare.js` tool will then produce a csv file with the benchmark results.
```bash
node benchmark/compare.js --old ./node-main --new ./node-pr-5134 string_decoder > compare-pr-5134.csv
```
_Tips: there are some useful options of `benchmark/compare.js`. For example,
if you want to compare the benchmark of a single script instead of a whole
module, you can use the `--filter` option:_
```console
--new ./new-node-binary new node binary (required)
--old ./old-node-binary old node binary (required)
--runs 30 number of samples
--filter pattern string to filter benchmark scripts
--exclude pattern excludes scripts matching <pattern> (can be
repeated)
--set variable=value set benchmark variable (can be repeated)
--no-progress don't show benchmark progress indicator
Examples:
--set CPUSET=0 Runs benchmarks on CPU core 0.
--set CPUSET=0-2 Specifies that benchmarks should run on CPU cores 0 to 2.
Note: The CPUSET format should match the specifications of the 'taskset' command
```
For analyzing the benchmark results, use [node-benchmark-compare][] or the R
scripts:
* `benchmark/compare.R`
* `benchmark/bar.R`
```console
$ node-benchmark-compare compare-pr-5134.csv # or cat compare-pr-5134.csv | Rscript benchmark/compare.R
confidence improvement accuracy (*) (**) (***)
string_decoder/string-decoder.js n=2500000 chunkLen=16 inLen=128 encoding='ascii' *** -3.76 % ±1.36% ±1.82% ±2.40%
string_decoder/string-decoder.js n=2500000 chunkLen=16 inLen=128 encoding='utf8' ** -0.81 % ±0.53% ±0.71% ±0.93%
string_decoder/string-decoder.js n=2500000 chunkLen=16 inLen=32 encoding='ascii' *** -2.70 % ±0.83% ±1.11% ±1.45%
string_decoder/string-decoder.js n=2500000 chunkLen=16 inLen=32 encoding='base64-ascii' *** -1.57 % ±0.83% ±1.11% ±1.46%
...
```
In the output, _improvement_ is the relative improvement of the new version,
hopefully this is positive. _confidence_ tells if there is enough
statistical evidence to validate the _improvement_. If there is enough evidence
then there will be at least one star (`*`), more stars is just better. **However
if there are no stars, then don't make any conclusions based on the
_improvement_.** Sometimes this is fine, for example if no improvements are
expected, then there shouldn't be any stars.
**A word of caution:** Statistics is not a foolproof tool. If a benchmark shows
a statistical significant difference, there is a 5% risk that this
difference doesn't actually exist. For a single benchmark this is not an
issue. But when considering 20 benchmarks it's normal that one of them
will show significance, when it shouldn't. A possible solution is to instead
consider at least two stars (`**`) as the threshold, in that case the risk
is 1%. If three stars (`***`) is considered the risk is 0.1%. However this
may require more runs to obtain (can be set with `--runs`).
_For the statistically minded, the script performs an [independent/unpaired
2-group t-test][t-test], with the null hypothesis that the performance is the
same for both versions. The confidence field will show a star if the p-value
is less than `0.05`._
The `compare.R` tool can additionally produce a box plot by using the
`--plot filename` option. In this case there are 48 different benchmark
combinations, and there may be a need to filter the csv file. This can be done
while benchmarking using the `--set` parameter (e.g. `--set encoding=ascii`) or
by filtering results afterwards using tools such as `sed` or `grep`. In the
`sed` case be sure to keep the first line since that contains the header
information.
```console
$ cat compare-pr-5134.csv | sed '1p;/encoding='"'"ascii"'"'/!d' | Rscript benchmark/compare.R --plot compare-plot.png
confidence improvement accuracy (*) (**) (***)
string_decoder/string-decoder.js n=2500000 chunkLen=16 inLen=128 encoding='ascii' *** -3.76 % ±1.36% ±1.82% ±2.40%
string_decoder/string-decoder.js n=2500000 chunkLen=16 inLen=32 encoding='ascii' *** -2.70 % ±0.83% ±1.11% ±1.45%
string_decoder/string-decoder.js n=2500000 chunkLen=16 inLen=4096 encoding='ascii' *** -4.06 % ±0.31% ±0.41% ±0.54%
string_decoder/string-decoder.js n=2500000 chunkLen=256 inLen=1024 encoding='ascii' *** -1.42 % ±0.58% ±0.77% ±1.01%
...
```
![compare tool boxplot](doc_img/compare-boxplot.png)
### Comparing parameters
It can be useful to compare the performance for different parameters, for
example to analyze the time complexity.
To do this use the `scatter.js` tool, this will run a benchmark multiple times
and generate a csv with the results. To see how to use this script,
run `node benchmark/scatter.js`.
```bash
node benchmark/scatter.js benchmark/string_decoder/string-decoder.js > scatter.csv
```
After generating the csv, a comparison table can be created using the
`scatter.R` tool. Even more useful it creates an actual scatter plot when using
the `--plot filename` option.
```console
$ cat scatter.csv | Rscript benchmark/scatter.R --xaxis chunkLen --category encoding --plot scatter-plot.png --log
aggregating variable: inLen
chunkLen encoding rate confidence.interval
16 ascii 1515855.1 334492.68
16 base64-ascii 403527.2 89677.70
16 base64-utf8 322352.8 70792.93
16 utf16le 1714567.5 388439.81
16 utf8 1100181.6 254141.32
64 ascii 3550402.0 661277.65
64 base64-ascii 1093660.3 229976.34
64 base64-utf8 997804.8 227238.04
64 utf16le 3372234.0 647274.88
64 utf8 1731941.2 360854.04
256 ascii 5033793.9 723354.30
256 base64-ascii 1447962.1 236625.96
256 base64-utf8 1357269.2 231045.70
256 utf16le 4039581.5 655483.16
256 utf8 1828672.9 360311.55
1024 ascii 5677592.7 624771.56
1024 base64-ascii 1494171.7 227302.34
1024 base64-utf8 1399218.9 224584.79
1024 utf16le 4157452.0 630416.28
1024 utf8 1824266.6 359628.52
```
Because the scatter plot can only show two variables (in this case _chunkLen_
and _encoding_) the rest is aggregated. Sometimes aggregating is a problem, this
can be solved by filtering. This can be done while benchmarking using the
`--set` parameter (e.g. `--set encoding=ascii`) or by filtering results
afterwards using tools such as `sed` or `grep`. In the `sed` case be
sure to keep the first line since that contains the header information.
```console
$ cat scatter.csv | sed -E '1p;/([^,]+, ){3}128,/!d' | Rscript benchmark/scatter.R --xaxis chunkLen --category encoding --plot scatter-plot.png --log
chunkLen encoding rate confidence.interval
16 ascii 1302078.5 71692.27
16 base64-ascii 338669.1 15159.54
16 base64-utf8 281904.2 20326.75
16 utf16le 1381515.5 58533.61
16 utf8 831183.2 33631.01
64 ascii 4363402.8 224030.00
64 base64-ascii 1036825.9 48644.72
64 base64-utf8 780059.3 60994.98
64 utf16le 3900749.5 158366.84
64 utf8 1723710.6 80665.65
256 ascii 8472896.1 511822.51
256 base64-ascii 2215884.6 104347.53
256 base64-utf8 1996230.3 131778.47
256 utf16le 5824147.6 234550.82
256 utf8 2019428.8 100913.36
1024 ascii 8340189.4 598855.08
1024 base64-ascii 2201316.2 111777.68
1024 base64-utf8 2002272.9 128843.11
1024 utf16le 5789281.7 240642.77
1024 utf8 2025551.2 81770.69
```
![compare tool boxplot](doc_img/scatter-plot.png)
### Running benchmarks on the CI
To see the performance impact of a pull request by running benchmarks on
the CI, check out [How to: Running core benchmarks on Node.js CI][benchmark-ci].
## Creating a benchmark
### Basics of a benchmark
All benchmarks use the `require('../common.js')` module. This contains the
`createBenchmark(main, configs[, options])` method which will setup the
benchmark.
The arguments of `createBenchmark` are:
* `main` {Function} The benchmark function,
where the code running operations and controlling timers should go
* `configs` {Object} The benchmark parameters. `createBenchmark` will run all
possible combinations of these parameters, unless specified otherwise.
Each configuration is a property with an array of possible values.
The configuration values can only be strings or numbers.
* `options` {Object} The benchmark options. Supported options:
* `flags` {Array} Contains node-specific command line flags to pass to
the child process.
* `byGroups` {Boolean} option for processing `configs` by groups:
```js
const bench = common.createBenchmark(main, {
groupA: {
source: ['array'],
len: [10, 2048],
n: [50],
},
groupB: {
source: ['buffer', 'string'],
len: [2048],
n: [50, 2048],
},
}, { byGroups: true });
```
* `combinationFilter` {Function} Has a single parameter which is an object
containing a combination of benchmark parameters. It should return `true`
or `false` to indicate whether the combination should be included or not.
`createBenchmark` returns a `bench` object, which is used for timing
the runtime of the benchmark. Run `bench.start()` after the initialization
and `bench.end(n)` when the benchmark is done. `n` is the number of operations
performed in the benchmark.
The benchmark script will be run twice:
The first pass will configure the benchmark with the combination of
parameters specified in `configs`, and WILL NOT run the `main` function.
In this pass, no flags except the ones directly passed via commands
when running the benchmarks will be used.
In the second pass, the `main` function will be run, and the process
will be launched with:
* The flags passed into `createBenchmark` (the third argument)
* The flags in the command passed when the benchmark was run
Beware that any code outside the `main` function will be run twice
in different processes. This could be troublesome if the code
outside the `main` function has side effects. In general, prefer putting
the code inside the `main` function if it's more than just declaration.
```js
'use strict';
const common = require('../common.js');
const { SlowBuffer } = require('node:buffer');
const configs = {
// Number of operations, specified here so they show up in the report.
// Most benchmarks just use one value for all runs.
n: [1024],
type: ['fast', 'slow'], // Custom configurations
size: [16, 128, 1024], // Custom configurations
};
const options = {
// Add --expose-internals in order to require internal modules in main
flags: ['--zero-fill-buffers'],
};
// `main` and `configs` are required, `options` is optional.
const bench = common.createBenchmark(main, configs, options);
// Any code outside main will be run twice,
// in different processes, with different command line arguments.
function main(conf) {
// Only flags that have been passed to createBenchmark
// earlier when main is run will be in effect.
// In order to benchmark the internal modules, require them here. For example:
// const URL = require('internal/url').URL
// Start the timer
bench.start();
// Do operations here
const BufferConstructor = conf.type === 'fast' ? Buffer : SlowBuffer;
for (let i = 0; i < conf.n; i++) {
new BufferConstructor(conf.size);
}
// End the timer, pass in the number of operations
bench.end(conf.n);
}
```
### Creating an HTTP benchmark
The `bench` object returned by `createBenchmark` implements
`http(options, callback)` method. It can be used to run external tool to
benchmark HTTP servers.
```js
'use strict';
const common = require('../common.js');
const bench = common.createBenchmark(main, {
kb: [64, 128, 256, 1024],
connections: [100, 500],
duration: 5,
});
function main(conf) {
const http = require('node:http');
const len = conf.kb * 1024;
const chunk = Buffer.alloc(len, 'x');
const server = http.createServer((req, res) => {
res.end(chunk);
});
server.listen(common.PORT, () => {
bench.http({
connections: conf.connections,
}, () => {
server.close();
});
});
}
```
Supported options keys are:
* `port` - defaults to `common.PORT`
* `path` - defaults to `/`
* `connections` - number of concurrent connections to use, defaults to 100
* `duration` - duration of the benchmark in seconds, defaults to 10
* `benchmarker` - benchmarker to use, defaults to the first available http
benchmarker
[autocannon]: https://github.com/mcollina/autocannon
[benchmark-ci]: https://github.com/nodejs/benchmarking/blob/HEAD/docs/core_benchmarks.md
[git-for-windows]: https://git-scm.com/download/win
[nghttp2.org]: https://nghttp2.org
[node-benchmark-compare]: https://github.com/targos/node-benchmark-compare
[t-test]: https://en.wikipedia.org/wiki/Student%27s_t-test#Equal_or_unequal_sample_sizes%2C_unequal_variances_%28sX1_%3E_2sX2_or_sX2_%3E_2sX1%29
[wrk]: https://github.com/wg/wrk

View File

@ -0,0 +1,38 @@
# How to write documentation for the Node.js project
This document refers to the Node.js API documentation that gets deployed to [nodejs.org/en/docs][]
and consists in a general reference on how to write and update such documentation.
## Style Guide
For a style guide on how to write or update the Node.js documentation refer to the [doc/README][] document.
## Building
There are a few different commands that you can use to build and view the documentation locally,
the simplest one being:
```bash
make docserve
```
This command builds the documentation, spins up a local server and provides you with a URL to
it that you can navigate to in order to view the built documentation.
For more build options refer to the [documentation building][building-the-documentation] documentation.
And for more details about the tooling used to build the documentation refer to
the [API Documentation Tooling][] document.
## Linting and Formatting
To make sure that your changes pass linting run the following command:
```bash
make lint-md
```
[API Documentation Tooling]: ./api-documentation.md
[building-the-documentation]: ../../BUILDING.md#building-the-documentation
[doc/README]: ../../doc/README.md
[nodejs.org/en/docs]: https://nodejs.org/en/docs/

View File

@ -0,0 +1,495 @@
# How to write a test for the Node.js project
## What is a test?
Most tests in Node.js core are JavaScript programs that exercise a functionality
provided by Node.js and check that it behaves as expected. Tests should exit
with code `0` on success. A test will fail if:
* It exits by setting `process.exitCode` to a non-zero number.
* This is usually done by having an assertion throw an uncaught Error.
* Occasionally, using `process.exit(code)` may be appropriate.
* It never exits. In this case, the test runner will terminate the test because
it sets a maximum time limit.
Add tests when:
* Adding new functionality.
* Fixing regressions and bugs.
* Expanding test coverage.
## Test directory structure
See [directory structure overview][] for outline of existing test and locations.
## How to write a good test
A good test should be written in a style that is optimial for debugging
when it fails.
In principle, when adding a new test, it should be placed in a new file.
Unless there is strong motivation to do so, refrain from appending
new test cases to an existing file. Similar to the reproductions we ask
for in the issue tracker, a good test should be as minimal and isolated as
possible to facilitate debugging.
A good test should come with comments explaining what it tries to test,
so that when it fails, other contributors can fix it with the full context
of its intention, and be able to modify it in good confidence when the context
changes.
## Test structure
Let's analyze this basic test from the Node.js test suite:
```js
'use strict'; // 1
const common = require('../common'); // 2
const fixtures = require('../common/fixtures'); // 3
// This test ensures that the http-parser can handle UTF-8 characters // 5
// in the http header. // 6
const assert = require('node:assert'); // 8
const http = require('node:http'); // 9
const server = http.createServer(common.mustCall((req, res) => { // 11
res.end('ok'); // 12
})); // 13
server.listen(0, () => { // 14
http.get({ // 15
port: server.address().port, // 16
headers: { 'Test': 'Düsseldorf' }, // 17
}, common.mustCall((res) => { // 18
assert.strictEqual(res.statusCode, 200); // 19
server.close(); // 20
})); // 21
}); // 22
// ... // 23
```
### **Lines 1-3**
```js
'use strict';
const common = require('../common');
const fixtures = require('../common/fixtures');
```
The first line enables strict mode. All tests should be in strict mode unless
the nature of the test requires that the test run without it.
The second line loads the `common` module. The [`common` module][] is a helper
module that provides useful tools for the tests. Some common functionality has
been extracted into submodules, which are required separately like the fixtures
module here.
Even if a test uses no functions or other properties exported by `common`,
the test should still include the `common` module before any other modules. This
is because the `common` module includes code that will cause a test to fail if
the test leaks variables into the global space. In situations where a test uses
no functions or other properties exported by `common`, include it without
assigning it to an identifier:
```js
require('../common');
```
### **Lines 5-6**
```js
// This test ensures that the http-parser can handle UTF-8 characters
// in the http header.
```
A test should start with a comment containing a brief description of what it is
designed to test.
### **Lines 8-9**
```js
const assert = require('node:assert');
const http = require('node:http');
```
The test checks functionality in the `node:http` module.
Most tests use the `node:assert` module to confirm expectations of the test.
The require statements are sorted in
[ASCII][] order (digits, upper
case, `_`, lower case).
### **Lines 11-22**
```js
const server = http.createServer(common.mustCall((req, res) => {
res.end('ok');
}));
server.listen(0, () => {
http.get({
port: server.address().port,
headers: { 'Test': 'Düsseldorf' },
}, common.mustCall((res) => {
assert.strictEqual(res.statusCode, 200);
server.close();
}));
});
```
This is the body of the test. This test is simple, it just tests that an
HTTP server accepts `non-ASCII` characters in the headers of an incoming
request. Interesting things to notice:
* If the test doesn't depend on a specific port number, then always use 0
instead of an arbitrary value, as it allows tests to run in parallel safely,
as the operating system will assign a random port. If the test requires a
specific port, for example if the test checks that assigning a specific port
works as expected, then it is ok to assign a specific port number.
* The use of `common.mustCall` to check that some callbacks/listeners are
called.
* The HTTP server closes once all the checks have run. This way, the test can
exit gracefully. Remember that for a test to succeed, it must exit with a
status code of 0.
## General recommendations
### Timers
Avoid timers unless the test is specifically testing timers. There are multiple
reasons for this. Mainly, they are a source of flakiness. For a thorough
explanation go [here](https://github.com/nodejs/testing/issues/27).
In the event a test needs a timer, consider using the
`common.platformTimeout()` method. It allows setting specific timeouts
depending on the platform:
```js
const timer = setTimeout(fail, common.platformTimeout(4000));
```
will create a 4-second timeout on most platforms but a longer timeout on slower
platforms.
### The _common_ API
Make use of the helpers from the `common` module as much as possible. Please
refer to the [common file documentation](https://github.com/nodejs/node/tree/HEAD/test/common)
for the full details of the helpers.
#### common.mustCall
One interesting case is `common.mustCall`. The use of `common.mustCall` may
avoid the use of extra variables and the corresponding assertions. Let's
explain this with a real test from the test suite.
```js
'use strict';
require('../common');
const assert = require('node:assert');
const http = require('node:http');
let request = 0;
let listening = 0;
let response = 0;
process.on('exit', () => {
assert.equal(request, 1, 'http server "request" callback was not called');
assert.equal(listening, 1, 'http server "listening" callback was not called');
assert.equal(response, 1, 'http request "response" callback was not called');
});
const server = http.createServer((req, res) => {
request++;
res.end();
}).listen(0, () => {
listening++;
const options = {
agent: null,
port: server.address().port,
};
http.get(options, (res) => {
response++;
res.resume();
server.close();
});
});
```
This test could be greatly simplified by using `common.mustCall` like this:
```js
'use strict';
const common = require('../common');
const http = require('node:http');
const server = http.createServer(common.mustCall((req, res) => {
res.end();
})).listen(0, common.mustCall(() => {
const options = {
agent: null,
port: server.address().port,
};
http.get(options, common.mustCall((res) => {
res.resume();
server.close();
}));
}));
```
**Note:** Many functions invoke their callback with an `err` value as the first
argument. It is not a good idea to simply pass `common.mustCall()` to those
because `common.mustCall()` will ignore the error. Use `common.mustSucceed()`
instead.
#### Countdown module
The common [Countdown module](https://github.com/nodejs/node/tree/HEAD/test/common#countdown-module)
provides a simple countdown mechanism for tests that require a particular
action to be taken after a given number of completed tasks (for instance,
shutting down an HTTP server after a specific number of requests).
```js
const Countdown = require('../common/countdown');
const countdown = new Countdown(2, () => {
console.log('.');
});
countdown.dec();
countdown.dec(); // The countdown callback will be invoked now.
```
#### Testing promises
When writing tests involving promises, it is generally good to wrap the
`onFulfilled` handler, otherwise the test could successfully finish if the
promise never resolves (pending promises do not keep the event loop alive).
Node.js automatically crashes - and hence, the test fails - in the case of an
`unhandledRejection` event.
```js
const common = require('../common');
const assert = require('node:assert');
const fs = require('node:fs').promises;
// Wrap the `onFulfilled` handler in `common.mustCall()`.
fs.readFile('test-file').then(
common.mustCall(
(content) => assert.strictEqual(content.toString(), 'test2'),
));
```
### Flags
Some tests will require running Node.js with specific command line flags set. To
accomplish this, add a `// Flags:` comment in the preamble of the
test followed by the flags. For example, to allow a test to require some of the
`internal/*` modules, add the `--expose-internals` flag.
A test that would require `internal/freelist` could start like this:
```js
'use strict';
// Flags: --expose-internals
require('../common');
const assert = require('node:assert');
const freelist = require('node:internal/freelist');
```
In specific scenarios it may be useful to get a hold of `primordials` or
`internalBinding()`. You can do so using
```bash
node --expose-internals -r internal/test/binding lib/fs.js
```
This only works if you preload `node:internal/test/binding` by command line
flag.
### Assertions
When writing assertions, prefer the strict versions:
* `assert.strictEqual()` over `assert.equal()`
* `assert.deepStrictEqual()` over `assert.deepEqual()`
When using `assert.throws()`, if possible, provide the full error message:
```js
assert.throws(
() => {
throw new Error('Wrong value');
},
/^Error: Wrong value$/, // Instead of something like /Wrong value/
);
```
In the case of internal errors, prefer checking only the `code` property:
```js
assert.throws(
() => {
throw new ERR_FS_FILE_TOO_LARGE(`${sizeKiB} Kb`);
},
{ code: 'ERR_FS_FILE_TOO_LARGE' },
// Do not include message: /^File size ([0-9]+ Kb) is greater than 2 GiB$/
);
```
### Console output
Output written by tests to stdout or stderr, such as with `console.log()` or
`console.error()`, can be useful when writing tests, as well as for debugging
them during later maintenance. The output will be suppressed by the test runner
(`./tools/test.py`) unless the test fails, but will always be displayed when
running tests directly with `node`. For failing tests, the test runner will
include the output along with the failed test assertion in the test report.
Some output can help debugging by giving context to test failures. For example,
when troubleshooting tests that timeout in CI. With no log statements, we have
no idea where the test got hung up.
There have been cases where tests fail without `console.log()`, and then pass
when its added, so be cautious about its use, particularly in tests of the I/O
and streaming APIs.
Excessive use of console output is discouraged as it can overwhelm the display,
including the Jenkins console and test report displays. Be particularly
cautious of output in loops, or other contexts where output may be repeated many
times in the case of failure.
In some tests, it can be unclear whether a `console.log()` statement is required
as part of the test (message tests, tests that check output from child
processes, etc.), or is there as a debug aide. If there is any chance of
confusion, use comments to make the purpose clear.
### ES.Next features
For performance considerations, we only use a selected subset of ES.Next
features in JavaScript code in the `lib` directory. However, when writing
tests, for the ease of backporting, it is encouraged to use those ES.Next
features that can be used directly without a flag in
[all maintained branches][]. [node.green][] lists available features
in each release, such as:
* `let` and `const` over `var`
* Template literals over string concatenation
* Arrow functions when appropriate
## Naming test files
Test files are named using kebab casing. The first component of the name is
`test`. The second is the module or subsystem being tested. The third is usually
the method or event name being tested. Subsequent components of the name add
more information about what is being tested.
For example, a test for the `beforeExit` event on the `process` object might be
named `test-process-before-exit.js`. If the test specifically checked that arrow
functions worked correctly with the `beforeExit` event, then it might be named
`test-process-before-exit-arrow-functions.js`.
## Imported tests
### Web platform tests
See [`test/wpt`](../../test/wpt/README.md) for more information.
## C++ unit test
C++ code can be tested using [Google Test][]. Most features in Node.js can be
tested using the methods described previously in this document. But there are
cases where these might not be enough, for example writing code for Node.js
that will only be called when Node.js is embedded.
### Adding a new test
The unit test should be placed in `test/cctest` and be named with the prefix
`test` followed by the name of unit being tested. For example, the code below
would be placed in `test/cctest/test_env.cc`:
```cpp
#include "gtest/gtest.h"
#include "node_test_fixture.h"
#include "env.h"
#include "node.h"
#include "v8.h"
static bool called_cb = false;
static void at_exit_callback(void* arg);
class EnvTest : public NodeTestFixture { };
TEST_F(EnvTest, RunAtExit) {
v8::HandleScope handle_scope(isolate_);
v8::Local<v8::Context> context = v8::Context::New(isolate_);
node::IsolateData* isolateData = node::CreateIsolateData(isolate_, uv_default_loop());
Argv argv{"node", "-e", ";"};
auto env = node::CreateEnvironment(isolateData, context, 1, *argv, 2, *argv);
node::AtExit(env, at_exit_callback);
node::RunAtExit(env);
EXPECT_TRUE(called_cb);
}
static void at_exit_callback(void* arg) {
called_cb = true;
}
```
Next add the test to the `sources` in the `cctest` target in node.gyp:
```console
'sources': [
'test/cctest/test_env.cc',
...
],
```
The only sources that should be included in the cctest target are
actual test or helper source files. There might be a need to include specific
object files that are compiled by the `node` target and this can be done by
adding them to the `libraries` section in the cctest target.
The test can be executed by running the `cctest` target:
```bash
make cctest
```
A filter can be applied to run single/multiple test cases:
```bash
make cctest GTEST_FILTER=EnvironmentTest.AtExitWithArgument
```
`cctest` can also be run directly which can be useful when debugging:
```bash
out/Release/cctest --gtest_filter=EnvironmentTest.AtExit\*
```
### Node.js test fixture
There is a [test fixture][] named `node_test_fixture.h` which can be included by
unit tests. The fixture takes care of setting up the Node.js environment
and tearing it down after the tests have finished.
It also contains a helper to create arguments to be passed into Node.js. It
will depend on what is being tested if this is required or not.
### Test coverage
To generate a test coverage report, see the
[Test Coverage section of the Building guide][].
Nightly coverage reports for the Node.js `main` branch are available at
<https://coverage.nodejs.org/>.
[ASCII]: https://man7.org/linux/man-pages/man7/ascii.7.html
[Google Test]: https://github.com/google/googletest
[Test Coverage section of the Building guide]: https://github.com/nodejs/node/blob/HEAD/BUILDING.md#running-coverage
[`common` module]: https://github.com/nodejs/node/blob/HEAD/test/common/README.md
[all maintained branches]: https://github.com/nodejs/lts
[directory structure overview]: https://github.com/nodejs/node/blob/HEAD/test/README.md#test-directories
[node.green]: https://node.green/
[test fixture]: https://github.com/google/googletest/blob/HEAD/docs/primer.md#test-fixtures-using-the-same-data-configuration-for-multiple-tests-same-data-multiple-tests