2 2228b436 2024-02-10 benni .\" Copyright (c) 2023 Benjamin Stürz
4 2228b436 2024-02-10 benni .\" Permission to use, copy, modify, and distribute this software for any
5 2228b436 2024-02-10 benni .\" purpose with or without fee is hereby granted, provided that the above
6 2228b436 2024-02-10 benni .\" copyright notice and this permission notice appear in all copies.
8 2228b436 2024-02-10 benni .\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
9 2228b436 2024-02-10 benni .\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
10 2228b436 2024-02-10 benni .\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
11 2228b436 2024-02-10 benni .\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
12 2228b436 2024-02-10 benni .\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
13 2228b436 2024-02-10 benni .\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
14 2228b436 2024-02-10 benni .\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
16 2228b436 2024-02-10 benni .Dd $Mdocdate$
17 2228b436 2024-02-10 benni .Dt LEXICGEN 1
20 2228b436 2024-02-10 benni .Nm lexicgen
21 2228b436 2024-02-10 benni .Nd lexic generator
22 2228b436 2024-02-10 benni .Sh SYNOPSIS
24 2228b436 2024-02-10 benni .Op Fl v Ar visibility
25 2228b436 2024-02-10 benni .Op Fl p Ar package
26 2228b436 2024-02-10 benni .Op Fl o Ar output-file
27 2228b436 2024-02-10 benni .Ar input-file
28 2228b436 2024-02-10 benni .Sh DESCRIPTION
30 2228b436 2024-02-10 benni is a clone of lex(1) for generating lexers using Lexic.
31 2228b436 2024-02-10 benni See the FORMAT section for information about the format.
33 2228b436 2024-02-10 benni The options are as follows:
34 2228b436 2024-02-10 benni .Bl -tag -width -Ds
35 2228b436 2024-02-10 benni .It Fl v Ar visibility
36 2228b436 2024-02-10 benni Change the visibility of the lexer-internal data structures and functions. (default: private)
37 2228b436 2024-02-10 benni .It Fl p Ar package
38 2228b436 2024-02-10 benni Set a Java package for the generated class.
39 2228b436 2024-02-10 benni .It Fl o Ar output-file
40 2228b436 2024-02-10 benni Write the output to
41 2228b436 2024-02-10 benni .Ar output-file
42 2228b436 2024-02-10 benni instead of a generated path.
44 2228b436 2024-02-10 benni .Ar output-file
45 2228b436 2024-02-10 benni is '-', then write to the standard output.
48 2228b436 2024-02-10 benni A valid file consists of three sections:
49 2228b436 2024-02-10 benni .Bl -bullet -compact
51 2228b436 2024-02-10 benni Declarations
53 2228b436 2024-02-10 benni Definitions
55 2228b436 2024-02-10 benni and Code (optional)
58 2228b436 2024-02-10 benni The declaration section can contain the following statements:
59 2228b436 2024-02-10 benni .Bl -tag -width -Ds
60 2228b436 2024-02-10 benni .It Sy %import Ar package
62 2228b436 2024-02-10 benni .Ar package .
63 2228b436 2024-02-10 benni .It Sy %eof Ar name
64 2228b436 2024-02-10 benni Declare an end of file token called
67 2228b436 2024-02-10 benni Note: Should only be specified once.
68 2228b436 2024-02-10 benni .It Sy %error Ar name
69 2228b436 2024-02-10 benni Declare an error token called
72 2228b436 2024-02-10 benni Note: Should only be specified once.
73 2228b436 2024-02-10 benni .It Sy name No = Sy syntax No ;
74 2228b436 2024-02-10 benni Define a variable called
78 2228b436 2024-02-10 benni that can be refered to from token definitions.
81 2228b436 2024-02-10 benni The definition section contains token definitions.
82 2228b436 2024-02-10 benni A token definition consists of a
85 2228b436 2024-02-10 benni .Ar syntax expression .
86 2228b436 2024-02-10 benni The name must consist of alphabetic characters.
88 2228b436 2024-02-10 benni The following syntax expressions are supported:
89 2228b436 2024-02-10 benni .Bl -tag -width -Ds
90 2228b436 2024-02-10 benni .It Sy ident
91 2228b436 2024-02-10 benni An identifier is a sequence of one or more alphabetic characters.
92 2228b436 2024-02-10 benni Match if the rule identified by
95 2228b436 2024-02-10 benni .It Qq Sy string
96 2228b436 2024-02-10 benni Match if the input matches all characters of
97 2228b436 2024-02-10 benni .Ar string .
98 2228b436 2024-02-10 benni .It Bq Sy range
99 2228b436 2024-02-10 benni Match, if any of the fragments match.
102 2228b436 2024-02-10 benni starts with the
104 2228b436 2024-02-10 benni character, then negate the matching result.
105 2228b436 2024-02-10 benni The range consists of fragments.
106 2228b436 2024-02-10 benni A fragment can be a single character, an escape sequence or a character range.
107 2228b436 2024-02-10 benni Only single-character escape sequences are supported.
108 2228b436 2024-02-10 benni A character range has the syntax a-b,
109 2228b436 2024-02-10 benni where a is the first character and b is the last.
110 2228b436 2024-02-10 benni .It Pq expr
113 2228b436 2024-02-10 benni Zero or one counts of
114 2228b436 2024-02-10 benni .Ar expr .
116 2228b436 2024-02-10 benni One or more counts of
117 2228b436 2024-02-10 benni .Ar expr .
119 2228b436 2024-02-10 benni Zero or more counts of
120 2228b436 2024-02-10 benni .Ar expr .
121 2228b436 2024-02-10 benni .It expr1 expr2
123 2228b436 2024-02-10 benni followed by
124 2228b436 2024-02-10 benni .Ar expr2 .
125 2228b436 2024-02-10 benni .It expr1 & expr2
126 2228b436 2024-02-10 benni Only match if both
131 2228b436 2024-02-10 benni .It expr1 \e expr2
132 2228b436 2024-02-10 benni Match only if
134 2228b436 2024-02-10 benni matches, and
137 2228b436 2024-02-10 benni .It expr1 | expr2
138 2228b436 2024-02-10 benni Match if either or both of
145 2228b436 2024-02-10 benni The code sections contains any arbitrary code that will be put into the generated file.
147 2228b436 2024-02-10 benni .Sh EXAMPLES
148 2228b436 2024-02-10 benni The following code is an example of a lexicgen file:
149 2228b436 2024-02-10 benni .Bd -literal
150 2228b436 2024-02-10 benni // Declarations
151 2228b436 2024-02-10 benni %import eu.bandm.tools.lexic.*
152 2228b436 2024-02-10 benni %import java.io.IOException
153 2228b436 2024-02-10 benni %import java.io.InputStreamReader
154 2228b436 2024-02-10 benni %import java.nio.charset.StandardCharsets
156 2228b436 2024-02-10 benni // Declare an end-of-file token called EOF.
159 2228b436 2024-02-10 benni // Declare an error token called Error.
160 2228b436 2024-02-10 benni %error Error
162 2228b436 2024-02-10 benni Integer = "0" | [1-9][0-9]*;
163 2228b436 2024-02-10 benni Ident = [a-zA-Z_][a-zA-Z_0-9]*;
166 2228b436 2024-02-10 benni // Definitions
168 2228b436 2024-02-10 benni Whitespace : [ \en\et\er\ef]+;
169 2228b436 2024-02-10 benni // A File can be either an integer, or an identifier.
170 2228b436 2024-02-10 benni File : Integer | Ident;
175 2228b436 2024-02-10 benni public static void main(String[] args) {
176 2228b436 2024-02-10 benni // The construct() function is generated by lexicgen and returns a Lexer.
177 2228b436 2024-02-10 benni var lexer = construct();
179 2228b436 2024-02-10 benni try (var rdr = new InputStreamReader(System.in, StandardCharsets.UTF_8)) {
180 2228b436 2024-02-10 benni TokenSource<String, TokenType> tokens = lexer
181 2228b436 2024-02-10 benni .lex(CodePointSource.read(rdr, e -> {}))
182 2228b436 2024-02-10 benni .removeTypes(TokenType.Whitespace);
183 2228b436 2024-02-10 benni while (true) {
184 2228b436 2024-02-10 benni var token = tokens.get();
185 2228b436 2024-02-10 benni if (token.getType() == TokenType.EOF)
187 2228b436 2024-02-10 benni System.out.println(token);
189 2228b436 2024-02-10 benni } catch (IOException e) {}
192 2228b436 2024-02-10 benni .Sh EXIT STATUS
194 2228b436 2024-02-10 benni .Sh SEE ALSO
196 2228b436 2024-02-10 benni .Sh AUTHORS
197 2228b436 2024-02-10 benni .An Benjamin Stürz Aq Mt benni@stuerz.xyz
