Next: , Previous: Incremental Parsing Framework, Up: Syntax Protocol


18.4.4.3 Lexical analysis

— Class: drei-syntax:lexer

Class precedence list: lexer, standard-object, slot-object, t

Slots:

The base class for all lexers.

— Class: drei-syntax:incremental-lexer

Class precedence list: incremental-lexer, lexer, standard-object, slot-object, t

A subclass of lexer which maintains the buffer in the form of a sequence of lexemes that is updated incrementally.

In the sequence of lexemes maintained by the incremental lexer, the lexemes are indexed by a position starting from zero.

— Generic Function: drei-syntax:nb-lexemes lexer

Return the number of lexemes in the lexer.

— Generic Function: drei-syntax:lexeme lexer pos

Given a lexer and a position, return the lexeme in that position in the lexer.

— Generic Function: drei-syntax:insert-lexeme lexer pos lexeme

Insert a lexeme at the position in the lexer. All lexemes following pos are moved to one position higher.

— Generic Function: drei-syntax:delete-invalid-lexemes lexer from to

Invalidate all lexemes that could have changed as a result of modifications to the buffer

— Generic Function: drei-syntax:inter-lexeme-object-p lexer object

This generic function is called by the incremental lexer to determine whether a buffer object is an inter-lexeme object, typically whitespace. Client code must supply a method for this generic function.

— Generic Function: drei-syntax:skip-inter-lexeme-objects lexer scan

This generic function is called by the incremental lexer to skip inter-lexeme buffer objects. The default method for this generic function increments the scan mark until the object after the mark is not an inter-lexeme object, or until the end of the buffer has been reached.

— Generic Function: drei-syntax:update-lex lexer start-pos end

This function is called by client code as part of the buffer-update protocol to inform the lexer that it needs to analyze the contents of the buffer at least up to the end mark of the buffer. start-pos is the position in the lexeme sequence at which new lexemes should be inserted.

— Generic Function: drei-syntax:next-lexeme lexer scan

This generic function is called by the incremental lexer to get a new lexeme from the buffer. Client code must supply a method for this function that specializes on the lexer class. It is guaranteed that scan is not at the end of the buffer, and that the first object after scan is not an inter-lexeme object. Thus, a lexeme should always be returned by this function.