I want to start looking at defining the language a bit more strictly. The firststatement to look at is the OPEN statement.
This can have multiple forms.
The first most simple form is only trying to do an OPEN and not checking for errors. If there are errors the program simply stops.
{{ OPEN '','SITE-FILE' TO SITE.FILE }}
The second form is using an else clause to handle errors.
{{ OPEN '','SITE-FILE' TO SITE.FILE ELSE SITE.FILE = '' }}
A future form would be to allow for blocks to be created using the OPEN statement.
{{ OPEN '','SITE-FILE' TO SITE.FILE THEN }}
File opened.
{{ END ELSE }}
File not opened.
{{ END }}
The OPEN statement requires 2 expressions and a variable. The else clause is optional. EXPR itself can contain commas so each token needs to be get parsed in correctly, taking care of the various brackets that allow commas and strings.
Which now that I'm writing this I've realized my parser doesn't actually work that way. I assume commas and spaces are break charaters which will mess with the way I parse out expressions.
I need to write a proper parser that will let me specify a pattern and return me back the capture groups. A regex could work but it wouldn't be enough, nesting will cause issues with regexes and the regex I wrote is already a bit slow.
OPEN (EXPR),(EXPR) TO (IDENTIFIER) THEN? (EXPR) ELSE? (EXPR)
Given an OPEN statement, I want to have a general subroutine that will take a string and try to see if it matches this structure and if it does, it should return back each piece. Having a general purpose function like that will clean up much of my code.
I need to handle:
Parens
Square Brackets
Angle Brackets
Double Quotes
Single Quotes
These things can allow for any level nesting so I need to keep a stack counter that will keep track of when these things open and close.I think a SCAN.FROM.TO might be a good function to write. I would pass in a string, the starting position and ending position that I want to parse out. This however wouldn't be dynamic enough for some things.