Updates the template parser to detect and ignore processing instruction syntax (e.g. `<? foo ?>` or `<? foo >`). Currently it is being printed out as text.
Fixes#34371.
Previously I intentionally kept the lexer as generic as possible so that block detection can happen later. However, since #62644 we detect blocks in the lexer so we might as well normalize them there so all the upstream code doesn't have to account for things like whitespace.
The lexer's isNamedEntityEnd function stopped scanning entity names
when encountering a digit character, causing 24 valid HTML named
entities with digits in their names (e.g. ¹, ½, ▓)
to be treated as plain text instead of decoded to their corresponding
Unicode characters.
Fixes#51323
Currently the HTML parser will stop parsing as soon as it hits an end character in the name of an attribute (e.g. `/` or `>`). This ends up being problematic with some third-party packages like Tailwind which uses a wider range of characters for its class names. While the characters are fine when inside the `class` attribute, our current parser behavior prevents users from setting those classes conditionally through `[class.]` bindings.
These changes adjust the parser to handle such cases.
Fixes#61671.
PR Close#62742
When we introduced blocks, we made a deliberate decision to treat the `@` character as a reserved character in case we need to use it for other syntax in the future. This meant that some common cases, like writing out an email address in the template, can be broken.
After some recent discussions we decided to relax the requirement and only treat `@` as a reserve character if it's followed by a character sequence that matches a known block.
PR Close#62644
There were 26 duplicated block tests in `lexer_spec.ts`, likely due to merge conflicts. These changes remove the duplicates while keeping the 6 tests that were different.
PR Close#62644
Enables the new `@let` syntax by default.
`@let` declarations are defined as:
1. The `@let` keyword.
2. Followed by one or more whitespaces.
3. Followed by a valid JavaScript name and zero or more whitespaces.
4. Followed by the `=` symbol and zero or more whitespaces.
5. Followed by an Angular expression which can be multi-line.
6. Terminated by the `;` symbol.
Example usage:
```
@let user = user$ | async;
@let greeting = user ? 'Hello, ' + user.name : 'Loading';
<h1>{{greeting}}</h1>
```
Fixes#15280.
PR Close#56715
Previously, autocompletions were not available in two main cases. We correct them.
1. Autocompletions immediately after `@` were usually not working, for example `foo @|`. We fix this by causing the lexer to not consider the `@` part of the text node.
2. Autocompletions such as `@\nfoo`, where a newline follows a bare `@`, were not working because the language service visitor considered us inside the subsequent text node. We fix this by adding a block name span for the block keyword, and special-case whether we are completing inside the name span. If we are, we don't continue to the following text node.
PR Close#52198
A lot of our tests are wrapped in `{}` which serves no purpose, aside from increasing the nesting level and, in some cases, causing confusion. The braces appear to be a leftover from a time when all tests were wrapped in a `function main() {}`. The function declaration was removed in #21053, but the braces remained, presumably because it was easier to search&replace for `function main()`, but not to remove the braces at the same time.
PR Close#52239
Updates the lexer to parse blocks as incomplete, instead of throwing errors. This will allow us to better handle them further down in the pipeline.
PR Close#52047
Enables the new `@` block syntax by default by removing the `enabledBlockTypes` flags. There are still some internal flags that allow special use cases to opt out of the block syntax, like during XML parsing and when compiling older libraries (see #51979).
PR Close#51994
Switches the syntax for blocks from `{#block}{/block}` to `@block {}` based on the feedback from the community.
Read more about the decision-making process in our blog: https://blog.angular.io/meet-angulars-new-control-flow-a02c6eee7843
The existing block types changed in the following ways:
**Conditional blocks:**
```html
<!-- Before -->
{#if cond}
Main content
{:else if otherCond}
Else if content
{:else}
Else content
{/if}
<!-- After -->
@if (cond) {
Main content
} @else if (otherCond) {
Else if content
} @else {
Else content
}
```
**Deferred blocks**
```html
<!-- Before -->
{#defer when isLoaded}
Main content
{:loading} Loading...
{:placeholder} <icon>pending</icon>
{:error} Failed to load
{/defer}
<!-- After -->
@defer (when isLoaded) {
Main content
} @loading {
Loading...
} @placeholder {
<icon>pending</icon>
} @error {
Failed to load
}
```
**Switch blocks:**
```html
<!-- Before -->
{#switch value}
{:case 1}
One
{:case 2}
Two
{:default}
Default
{/switch}
<!-- After -->
@switch (value) {
@case (1) {
One
}
@case (2) {
Two
}
@default {
Default
}
}
```
**For loops**
```html
<!-- Before -->
{#for item of items; track item}
{{item.name}}
{:empty} No items
{/for}
<!-- After -->
@for (item of items; track item) {
{{item.name}}
} @empty {
No items
}
```
PR Close#51891
Fixes that using braces in the block parameters would result in incorrect tokens being produced. Currently we don't have any blocks that allow object literal parameters, but it may come up in the future.
PR Close#51143
⚠️Disclaimer⚠️ this PR implements syntax that is still in an open RFC. It will be adjusted once the RFC is closed.
These changes extend the lexer to recognize the concepts of a block group (`{#foo paramA; paramB}{/foo}`) and a block (`{:foo paramA; paramB;}`) which will be useful later on for the control flow and defer proposals. Block groups can be used anywhere and require a closing tag while block can only be used inside of a block.
The idea is that in the next PRs the markup AST will be expanded to have some more specialized node like `ConditionalBlock` or `DeferBlock` which will then be turned into instructions.
PR Close#50895
These token interfaces will make it easier to reason about tokens in the
parser and in specs.
Previously, it was never clear what items could appear in the `parts`
array of a token given a particular `TokenType`. Now, each token interface
declares a labelled tuple for the parts, which helps to document the token
better.
PR Close#43132
The lexer now splits encoded entity tokens out from text and attribute value tokens.
Previously encoded entities would be decoded and the decoded value would be
included as part of the text token of the surrounding text. Now the entities
have their own tokens. There are two scenarios: text and attribute values.
Previously the contents of `<div>Hello & goodbye</div>` would be a single
TEXT token. Now it will be three tokens:
```
TEXT: "Hello "
ENCODED_ENTITY: "&", "&"
TEXT: " goodbye"
```
Previously the attribute value in `<div title="Hello & goodbye">` would be
a single text token. Now it will be three tokens:
```
ATTR_VALUE_TEXT: "Hello "
ENCODED_ENTITY: "&", "&"
ATTR_VALUE_TEXT: " goodbye"
```
- ENCODED_ENTITY tokens have two parts: "decoded" and "encoded".
- ENCODED_ENTITY tokens are always preceded and followed by either TEXT tokens
or ATTR_VALUE_TEXT tokens, depending upon the context, even if they represent
an empty string.
The HTML parser has been modified to recombine these tokens to allow this
refactoring to have limited effect in this commit. Further refactorings
to use these new tokens will follow in subsequent commits.
PR Close#43132
The lexer now splits interpolation tokens out from text tokens.
Previously the contents of `<div>Hello, {{ name}}<div>` would be a single
text token. Now it will be three tokens:
```
TEXT: "Hello, "
INTERPOLATION: "{{", " name", "}}"
TEXT: ""
```
- INTERPOLATION tokens have three parts, "start marker", "expression"
and "end marker".
- INTERPOLATION tokens are always preceded and followed by TEXT tokens,
even if they represent an empty string.
The HTML parser has been modified to recombine these tokens to allow this
refactoring to have limited effect in this commit. Further refactorings
to use these new tokens will follow in subsequent commits.
PR Close#43132
These token interfaces will make it easier to reason about tokens in the
parser and in specs.
Previously, it was never clear what items could appear in the `parts`
array of a token given a particular `TokenType`. Now, each token interface
declares a labelled tuple for the parts, which helps to document the token
better.
PR Close#42062
Previously, the way templates were tokenized meant that we lost information
about the location of interpolations if the template contained encoded HTML
entities. This meant that the mapping back to the source interpolated strings
could be offset incorrectly.
Also, the source-span assigned to an i18n message did not include leading
whitespace. This confused the output source-mappings so that the first text
nodes of the message stopped at the first non-whitespace character.
This commit makes use of the previous refactorings, where more fine grain
information was provided in text tokens, to enable the parser to identify
the location of the interpolations in the original source more accurately.
Fixes#41034
PR Close#42062
The lexer now splits encoded entity tokens out from text and attribute value tokens.
Previously encoded entities would be decoded and the decoded value would be
included as part of the text token of the surrounding text. Now the entities
have their own tokens. There are two scenarios: text and attribute values.
Previously the contents of `<div>Hello & goodbye</div>` would be a single
TEXT token. Now it will be three tokens:
```
TEXT: "Hello "
ENCODED_ENTITY: "&", "&"
TEXT: " goodbye"
```
Previously the attribute value in `<div title="Hello & goodbye">` would be
a single text token. Now it will be three tokens:
```
ATTR_VALUE_TEXT: "Hello "
ENCODED_ENTITY: "&", "&"
ATTR_VALUE_TEXT: " goodbye"
```
- ENCODED_ENTITY tokens have two parts: "decoded" and "encoded".
- ENCODED_ENTITY tokens are always preceded and followed by either TEXT tokens
or ATTR_VALUE_TEXT tokens, depending upon the context, even if they represent
an empty string.
The HTML parser has been modified to recombine these tokens to allow this
refactoring to have limited effect in this commit. Further refactorings
to use these new tokens will follow in subsequent commits.
PR Close#42062
The lexer now splits interpolation tokens out from attribute value tokens.
Previously the attribute value of `<div attr="Hello, {{ name}}">` would be a single
token. Now it will be three tokens:
```
ATTR_VALUE_TEXT: "Hello, "
ATTR_VALUE_INTERPOLATION: "{{", " name", "}}"
ATTR_VALUE_TEXT: ""
```
- ATTR_VALUE_INTERPOLATION tokens have three parts, "start marker",
"expression" and "end marker".
- ATTR_VALUE_INTERPOLATION tokens are always preceded and followed
by TEXT tokens, even if they represent an empty string.
The HTML parser has been modified to recombine these tokens to allow this
refactoring to have limited effect in this commit. Further refactorings
to use these new tokens will follow in subsequent commits.
PR Close#42062
The lexer now splits interpolation tokens out from text tokens.
Previously the contents of `<div>Hello, {{ name}}<div>` would be a single
text token. Now it will be three tokens:
```
TEXT: "Hello, "
INTERPOLATION: "{{", " name", "}}"
TEXT: ""
```
- INTERPOLATION tokens have three parts, "start marker", "expression"
and "end marker".
- INTERPOLATION tokens are always preceded and followed by TEXT tokens,
even if they represent an empty string.
The HTML parser has been modified to recombine these tokens to allow this
refactoring to have limited effect in this commit. Further refactorings
to use these new tokens will follow in subsequent commits.
PR Close#42062
Previously the lexer would break out of consuming a text token if it contains
a `<` character. Then if the next characters did not indicate an HTML syntax
item, such as a tag or comment, then it would start a new text token. These
consecutive text tokens are then merged into each other in a post tokenization
step.
In the commit before this, interpolation no longer leaks across text tokens.
The approach given above to handling `<` characters that appear in text is
no longer adequate. This change ensures that the lexer only breaks out of
a text token if the next characters indicate a valid HTML tag, comment,
CDATA etc.
PR Close#42605
When consuming a text token, the lexer tracks whether it is reading characters
from inside an interpolation so that it can identify invalid ICU expressions.
Inside an interpolation there will be no ICU expression so it is safe to
have unmatched `{` characters, but outside an interpolation this is an error.
Previously, if an interpolation was started, by an opening marker (e.g. `{{`)
in a text token but the text came to an end before the closing marker (e.g. `}}`)
then the lexer was not clearing its internal state that tracked that it was
inside an interpolation. When the next text token was being consumed,
the lexer, incorrectly thought it was already within an interpolation.
This resulted in invalid ICU expression errors not being reported.
For example, in the following snippet, the first text block has a prematurely
ended interpolation, and the second text block contains an invalid `{` character.
```
<div>{{</div>
<div>{</div>
```
Previously, the lexer would not have identified this as an error. Now there
will be an EOF error that looks like:
```
TS-995002: Unexpected character "EOF"
(Do you have an unescaped "{" in your template? Use "{{ '{' }}") to escape it.)
```
PR Close#42605
The compiler's parsing code has logic to recover from incomplete open
tags (i.e. `<div`) but the recovery logic does not handle when the
incomplete tag is terminated by an EOF. This commit updates the logic to
allow for the EOF character to be interpreted as the end of the tag open
so that the parser can continue processing. It will then fail to find
the end tag and recover by marking the open tag as incomplete.
Part of https://github.com/angular/vscode-ng-language-service/issues/1140
PR Close#41054
Kristiyan Kostadinov