RosettaCodeData/Task/Tokenize-a-string-with-escaping/TMG/tokenize-a-string-with-escaping.tmg
2023-07-01 13:44:08 -04:00

15 lines
348 B
Text

prog: char(sep) *
char(esc) *
str: smark
token: forw/outp
( [ch==esc?] char(ch) any(!<<>>) token
| [ch==sep?] char(ch) outp str
| any(!<<>>) token );
outp: parse(( scopy = { <"> 1 <"> * } ));
forw: peek/chkeof;
peek: [ch=0] char(ch) fail;
chkeof: ( [ch?] succ | fail );
ch: 0;
sep: 0;
esc: 0;