tokenizer
A grammar describes the syntax of a programming language, and might be defined in Backus-Naur form (BNF). A lexer performs lexical analysis, turning text into tokens. A parser takes tokens and builds a data structure like an abstract syntax tree (AST). The parser is concerned with context: does the sequence of tokens fit the grammar? A compiler is a combined lexer and parser, built for a specific grammar.
Here are 689 public repositories matching this topic...
-
Updated
Oct 1, 2021 - Python
-
Updated
Oct 31, 2021 - Python
-
Updated
May 14, 2018 - Swift
-
Updated
Feb 11, 2022 - JavaScript
-
Updated
Mar 18, 2022 - Go
-
Updated
Feb 3, 2021 - Java
-
Updated
Feb 8, 2021 - Python
-
Updated
Mar 1, 2021 - Scala
-
Updated
Mar 12, 2022 - Python
-
Updated
Mar 1, 2022 - Java
-
Updated
Feb 26, 2022 - Java
Prepare the parser to PHP 8 :
Work In Progress as RFC are not yet closed
-
Updated
Mar 15, 2022 - Python
-
Updated
Aug 30, 2021 - C
Would it be possible to have the regex parser support character classes like \w
within other character classes? I had a regex pattern earlier that used the character class [0-9a-zA-Z_\.-]
, and I attempted to simplify it with [\w\.\-]
. I didn't notice this library doesn't support doing that, and was wondering just how difficult that would be to implement. For the time being i'm just expanding
-
Updated
Jun 18, 2021 - Go
-
Updated
Oct 4, 2021 - C++
-
Updated
Mar 22, 2022 - TypeScript
-
Updated
Mar 22, 2022 - Python
-
Updated
Feb 16, 2022 - C++
-
Updated
Dec 4, 2021 - C++
-
Updated
Mar 6, 2022 - Python
-
Updated
Jul 14, 2021 - R
-
Updated
Mar 7, 2022 - C++
-
Updated
Mar 12, 2022 - Python
-
Updated
Mar 22, 2022 - Rust
- Wikipedia
- Wikipedia
Everything in diagrams.css should be scoped to some wrapping css class, because as is it cannot be bundled with the rest of an app's css because of styles like this:
Curre