Remove re-exports of `definePlugin`, `defineRule` and `RuleTester` from main `index` entry point of `oxlint` package.
We want this to be cheap:
```js
import { defineConfig } from "oxlint";
```
But with the deprecated exports, this would eagerly load ~20,000 lines of JS plugins-related code. See `index.js` in `dist` directory before and after this PR in release build. It was the import from `./lint.js` which was massive.
Personally, I don't think we need to keep the deprecated exports, for 3 reasons:
1. We have very clearly said that JS plugins are experimental and that breaking changes are very possible.
2. Given how broken our types are at present, probably few people were using `definePlugin` and `defineRule` anyway!
3. The fix for users who do get broken is trivial:
```diff
- import { definePlugin } from "oxlint";
+ import { definePlugin } from "@oxlint/plugins";
```
Part of #18610.
Our intent is to move `RuleTester` to a separate package `@oxlint/plugins-dev`.
So rename the current entry point which exports `RuleTester` from `oxlint/rule-tester` to `oxlint/plugins-dev`.
Part of #18610.
Our intent is to move `definePlugin`, `defineRule`, and `eslintCompatPlugin` to a separate package `@oxlint/plugins` (plural to avoid confusion with naming convention for plugins `oxlint-plugin-xxx`).
So rename the current entry point which exports these functions from `oxlint/plugin` to `oxlint/plugins` (plural).
This is technically breaking (only for types), but custom plugins are experimental.
The ruletester/defineRule/definePlugin are re-exported but with a `@deprecated` tag which I think is a good enough trade off
I want to avoid users bringing in a ton of custom plugin code when doing `import { defineConfig } from 'oxlint'`
Previously we used `@typescript-eslint/typescript-estree` parser to tokenize a source text. `@typescript-eslint/typescript-estree` uses TypeScript parser internally, and then builds the tokens array by traversing TypeScript's AST.
Instead, use TypeScript parser directly, and copy the code from TS-ESLint for converting tokens from TypeScript's format to TS-ESTree format.
### Why?
This has a few advantages:
1. Remove a dependency.
2. TS-ESLint's tokens did not have `start` or `end` properties, which we want for consistency with our AST. By reimplementing ourselves, we can add those properties to tokens.
3. TS-ESLint does a bunch of other work besides compiling the tokens array - it converts the AST too. We discard the AST that TS-ESLint produces, so this work is pointless. By cutting out TS-ESLint, we also cut out that unnecessary work, and do only what we need to - tokenizing.
4. TS-ESLint would throw an error if TypeScript produces any errors (even non-fatal ones), and there is no option to tell TS-ESLint not to do that. This caused a bunch of conformance tests to fail, because ESLint's tests include invalid code.
5. Can add workarounds for TypeScript bugs - e.g. not being able to parse JSX closing fragments containing line breaks (`<>hello<\n/>`).
### Implementation detail
We no longer need to get comments from TypeScript. We use the comments produced by Oxc parser instead.
### `CommentToken` type
A side effect is that there's no longer any need for separate `Comment` and `CommentToken` types - as we use our own comments now. So this PR removes `CommentToken` type.
Have marked this as a breaking change due to removal of this type.
Fix TS types for `RuleTester`. Merging the `RuleTester` class and namespace was working OK within the repo, but not in the built bundle. Fix that by defining the namespace in same file as the class.
Improve the type def for `Token`.
Previously `Token` included `CommentToken`. This wasn't quite right because the `tokens` property of `Program` is `Token[]`, but it doesn't include comments.
So make `Token` the type for just code tokens, and use `Token | CommentToken` everywhere where the value can be either a code token or a comment token.
Also, remove the `BlockCommentToken` and `LineCommentToken` types. It seems unnecessarily detailed. `CommentToken` now covers both with `type: "Line" | "Block"`.
Part of #16018.
Implement `RuleTester` class for testing rules. Has same API as ESLint's `RuleTester`.
A few features not implemented yet:
* Fixes are not executed, or checked against provided `output`.
* Suggestions are not checked (we don't yet have support for suggestions in JS plugins).
* No use of config provided through `RuleTester` class constructor or via `RuleTester.setDefaultConfig`.
Preparatory work for supporting rule options.
Store options for all rules in an `allOptions` array. Currently this array only contains a single value for empty options, but in future we'll populate it with options extracted from configs, sent over from Rust side.
Am adding this now, before full implementation (#14825), as it's required for rule tester.
Rename assertion functions to clarify that they're not "real" assertions which we can rely on for validating input, and are only for compile-time testing or debug assertions.
`typeAssertIs` is not a great name, but I couldn't come up with anything better. `debugAssertIs` would be not be accurate, as it'd imply that in debug builds in performs an actual assertion, which it doesn't.
In any case, we should replace `typeAssertIs` with specialized assertion functions which do actually perform runtime assertions in debug builds (e.g. `typeAssertIs<Fix>()` -> `debugAssertIsFix()`).
- "Slow but working" implementation for
https://github.com/oxc-project/oxc/issues/14829#issuecomment-3512679496
- Uses the parser from `@typescript-eslint/typescript-estree` for
tokens. The source code is fully parsed and everything but the tokens
and comments is discarded.
- Tests are directly adapted from `eslint`.
- Direct commits from maintainers welcome.
### Tasks
- [x] `getTokens`
- [x] Move token types to `src-js/plugins/tokens.ts`
- [x] Bundle typescript, handling `_filename` usage in ES modules.
- [x] ~[Prevent class name mangling by
`tsdown`](https://github.com/oxc-project/oxc/pull/15861#discussion_r2542816947).
Could be deferred.~
https://github.com/oxc-project/oxc/pull/15906#pullrequestreview-3487271352
- [x] [Test regex-divide ambiguous
syntax](https://github.com/oxc-project/oxc/pull/15861#discussion_r2542919409).
- [x] Test for conformance directly against `eslint` tests.
- [ ] [Parse as JSX only if source text is
JSX](https://github.com/oxc-project/oxc/pull/15861#discussion_r2542850640).
Could be deferred.
- [ ] Lazy load the parser on demand. Could be deferred.
### Future work
- [ ] `getFirstToken`
- [ ] `getFirstTokens`
- [ ] `getLastToken`
- [ ] `getLastTokens`
- [ ] `getTokenBefore`
- [ ] `getTokenOrCommentBefore`
- [ ] `getTokensBefore`
- [ ] `getTokenAfter`
- [ ] `getTokenOrCommentAfter`
- [ ] `getTokensAfter`
- [ ] `getTokensBetween`
- [ ] `getFirstTokenBetween`
- [ ] `getFirstTokensBetween`
- [ ] `getLastTokenBetween`
- [ ] `getLastTokensBetween`
- [ ] `getTokenByRangeStart`
- [ ] `isSpaceBetween`
- [ ] `isSpaceBetweenTokens`
### Decisions
- [x] `@typescript-eslint/typescript-estree` peer-depends on
`typescript`. How should we package it? Currently, `typescript` is being
made an optional dependency.
- **overlookmotel (on discord): ideally bundled, direct runtime
dependency otherwise.**
- [x] Deprecated methods are being removed altogether in ESLint 10:
`getTokenOrCommentBefore`, `getTokenOrCommentAfter`, and
`isSpaceBetweenTokens`. These are surface level deprecations: the
functionality was merged with other methods (the `includeComments: true`
option) and plugins can migrate with a one line change. I'm guessing we
are targeting fairly modern, actively developed projects. Should we
expose them?
- **[overlookmotel and
camc314](https://github.com/oxc-project/oxc/pull/15861#issuecomment-3552651295):
yes.**
- [x] `Program` range is different between ESLint and TS-ESLint. Which
one should we follow?
-
**[overlookmotel](https://github.com/oxc-project/oxc/pull/15861#discussion_r2547915407):
they are going to both change and align soon, we should update our range
calculation to match when they do.**
---------
Co-authored-by: overlookmotel <theoverlookmotel@gmail.com>
Pure refactor. Split up `plugins/utils.ts` into 3 files in `utils` directory.
Most importantly, assertion functions are by themselves in an `asserts.ts` file. This paves the way for the next PR (#15946).
Adapted from #15868. Enable `strictNullChecks` in TSConfig for `apps/oxlint`.
Rather than using `!` all over the place, I've added a function `assertIsNonNull`, which performs the same function. Idea is that we can later on produce a debug build where `assertIsNonNull` does an actual runtime assertion (rather than just a TS `asserts`) that the value is not `null` or `undefined`. We can use the debug build in tests, and catch any logic errors.
Unfortunately, TypeScript doesn't seem able to track side effects, so there seems to be no way to tell it "after calling `initAst()`, `ast` is always non-null". Don't want to switch to a having a `getAst` function which returns a non-null value, as that'd introduce unnecessary temp vars, and function calls, on hot paths. So this seems to be the best we can do without incurring runtime cost.
Small perf optimization. In ESLint compat mode, `create` returns an empty visitor object when `before` hook returns `false`. Reuse the same object, instead of creating a new one each time.
Improve `Diagnostic` and `Suggestion` types, by using `RequireAtLeastOne` utility type from `type-fest` package, to enforce that:
* Either `message` or `messageId` property must be provided.
* Either `node` or `loc` property must be provided.
Change to TSDown config `dts: { resolve: true }` is get it to bundle `type-fest` into the `.d.ts` file.
Add `Suggestion` type, and `suggest` field to `Diagnostic`.
Suggestions are not implemented yet, but this completes the interface for diagnostics defined by ESLint's docs.
Fill out the TS type def for `RuleMeta`, based on [ESLint's docs](https://eslint.org/docs/latest/extend/custom-rules).
Most of the properties can be ignored at runtime in Oxlint (they're primarily useful for generating docs), but this has also surfaced a few features we need to implement. These are marked with "TODO" comments, and I'll also open issues for them shortly.
Implement 4 methods on `Context` which are present in ESLint, but deprecated.
* `getFilename`
* `physicalFilename`
* `getCwd`
* `getSourceCode`
After some discussion, @camc314 and I concluded that we should add these methods despite their deprecation. The reasons behind that decision are documented in a comment in this PR.
Revert #15447. It's easy to allow `Context#id` to be accessed in `createOnce` when running rule in Oxlint, but not possible to do in ESLint compat mode with current API, because `id` isn't a property of the rule, so `defineRule` doesn't know it.
We could support it by making `definePlugin` compulsory for ESLint interop, and get it to set `id` property on `Context` objects properly.
But that's a bit tricky, so just disable access for now. It's probably not very useful anyway - a rule should know what its own name is!
#15301 made `Context#cwd` available in `createOnce`. Make it available in ESLint compat mode too. Ditto the `extend` method added in #15477.
Define all other `Context` methods/getters on prototype of `Context` in ESLint compat mode, for completeness.
It makes the types a bit shit, but so far we accept `null` or `undefined` for all properties, and treat them the same as the property being absent. Extend this convention to `message` and `messageId` properties.
Also split out getting the message from a diagnostic into a separate function, to make this pattern simpler via early return.
Add `loc` to all AST nodes.
`loc` is computed lazily, as it's quite an expensive operation - because it requires searching the source text for line breaks and building a translation table.
We can consider making `loc` a "real" property once we have a faster translation table built on Rust side.
Export AST types directly from `oxlint` package, instead of depending on `@oxc-project/types`. This enables altering the types to reflect that `range` field is always present on AST node in Oxlint's AST (#14354).
Add `range` field to AST nodes, using the support for `range` in raw transfer added in #14319.
Use `range` field rather than `start` + `end` in nodes passed to `context.report` and fixer methods - for closer alignment with ESLint.
Add support for ESLint's `SourceCode` API. At present the only method on `SourceCode` is `getText`. More methods will be added in further PRs to follow.
overlookmotel