Questions tagged [parsing]
Analyzing (un)structured data to convert it into a structured, normalized format.
294
questions
-1
votes
1
answer
64
views
Is it a good idea to let keywords have different lexical rules from names of types, variables, functions, etc? [closed]
For example, keywords have a special prefix. Objective-C has @interface, @implementation, but that's for compatibility with C. It inherits all the C keywords of course, with no @.
How about a language ...
- 117
-1
votes
2
answers
127
views
When writing a tokenizer, what is the standard practice for handling aliased language keywords?
When writing a tokenizer, what is the standard practice for handling aliased language keywords?
For example, notethat signed short int is a language keyword in C++ and several aliases might be allowed....
- 145
-1
votes
1
answer
149
views
Better way to represent grammar symbols in C
I'm trying to build a simple compiler for a subset of the C language in C. To achieve this, I needed to figure out a way to represent the grammar symbols. Basically, each symbol can either be a "...
- 147
0
votes
1
answer
95
views
How is it possible to store the AST nodes location in the source code?
I created a simple parser in Rust and defined the AST like this:
enum Expression {
Number(i32),
BinaryOperator(Box<Expression>, Operator, Box<Expression>),
Identifier(String),
}...
- 111
1
vote
3
answers
460
views
Parse 8 bytes to date time
I am trying to parse a file created by another software, but I cant identify a pattern on how this datetime is saved. There doesnt seem to be any consistency.
Programming language of the software is C+...
- 135
-2
votes
2
answers
99
views
How to filter and concatenate multiple sql files into one database [closed]
I have an issue where I have multiple databases from previous projects that I would like to combine into one large database. These databases are stored in .sql files. The issue is that I only need ...
- 7
1
vote
3
answers
195
views
Using source code instead of XML/JSON or other custom serialization schemes and binary file formats
For a while now I have been toying with the idea of using source code as a file storage format.
My question: How to support format version changes? (loading of older files with structural differences)
...
- 127
0
votes
0
answers
153
views
What is it about kdb/q that makes the grammar not suitable for ANTLR style parser generators?
I want to build a code analysis tool for personal use when programming in kdb/q.
In order to do this, I need to be able to parse q code into an AST. I have never written a parser before. ANTLR4 seems ...
- 127
1
vote
2
answers
99
views
What do you call a process which transforms objects of complex types into simple objects of primitive types? [closed]
My first thought was that I'm "serializing" the complex object, but from what I understand that means I'm reducing it down to a string or binary format which could be passed over a network. ...
0
votes
1
answer
80
views
Parsing complex data type names
How do compilers parse complex data types names like function pointers. The type has to be somehow put into the AST or it has to be processed during parsing. What are the pros and cons of different ...
- 13
-1
votes
2
answers
193
views
What would be the performant approach to parsing a _limited subset_ of Markdown? [closed]
There are plenty of articles about parsing full Markdown/CommonMark/etc, but I'm interested only in a specific limited subset, so I'm wondering if a better approach is available to me, and would much ...
- 125
-1
votes
1
answer
381
views
How does a lexer handle template strings?
So lexers are supposed to emit tokens for key structures like INDENT and DEDENT for indentation stuff, or these:
NUMBER ::= [0-9]+
ID ::= [a-Z]+, except for keywords
IF ::= 'if'
LPAREN ::= '('
...
- 2,467
0
votes
3
answers
539
views
Is it possible to make a compiler for any dynamic/script/interpreter language
Logically, not based on how cost we will spend or how much we will hire programmers to do it.
Can we (Is it possible to) make a compiler for any dynamic/script/interpreter language, like Lua, Python, ...
- 19
8
votes
1
answer
362
views
What type of syntax notation is this?
SQL Server documentation uses this notation, which is very easy to understand and consume. Is this a BNF Syntax Diagram? Or is this a different type of notation?
Source: SQL server documentation page ...
- 259
4
votes
2
answers
1k
views
How does the GLL parsing algorithm work?
I'm very interested in the topic of parsers, especially in the topic of parser combinators like Superpower. The problem with them is that the grammars that they can work with are a bit limited. For ...
- 413
0
votes
1
answer
252
views
Benefits of using a tokenizer/lexer before parsing for recusive descent parser
I am trying to build a static program analyzer for a proprietary progamming language for a school project, and am currently trying to implement the parser from scratch.
I was wondering, what are the ...
- 101
15
votes
3
answers
3k
views
How would you test a lexer?
I'm wondering how to effectively test a lexer (tokenizer).
The number of combinations of tokens in a source file can be huge, and the only way I've found is to make a batch of representative source ...
- 413
1
vote
0
answers
76
views
Is there a easy and useful error handling algorithm for bottom-up based parser?
My English skill is poor because I'm not a native English speaker.
Please understand.
I wonder that there is a error handling algorithm easy and useful in LR parser.
LR Parser is bottom up based so it ...
- 207
7
votes
3
answers
3k
views
What are the 'practical' advantages of LR parser over LL parser 'in today'?
My English skill is poor because I'm not a native English speaker.
Please understand.
I write this article because want to discuss about this topic.
I think LR parser has no 'practical' advantages ...
- 207
6
votes
5
answers
614
views
Reasons to use (and not to use) a repeated delimiter to escape that delimiter?
For the designer of a language syntax, what are some reasons to choose a repeated delimiter to escape that delimiter, instead of having a separate escape character to escape that delimiter. A common ...
- 171
2
votes
1
answer
126
views
Ask for suggestion: data type for parsing stringified fractional numbers
I am the author of a C library for parsing INI files. So far I have delegated the task of parsing values as numbers to the standard atoi() family of functions. However I think time has come that I ...
- 129
-3
votes
1
answer
250
views
Data structures or coding styles in C++ for avoiding long elseif chain when parsing?
Lately I have created some small parsers of data. My initial code structure
// more cases here ...
else if(!strcmp(X,"somekey")){
// Parse according to "somekey" behavior.
}
// ...
- 200
1
vote
3
answers
177
views
Why are code readability and debugging arguments often expressed as a counter-argument for the use of generated LR parsers?
When it comes to using an LR parser generated by a tool, such as Bison, a disadvantage that often comes up as counterarguments is that the resulting parser will be unreadable and complicated to debug, ...
- 219
0
votes
0
answers
223
views
How can you multithread an html parser [ in C++ and similar languages ]?
Ive done two HTML parsers.
Done with Regular Expressions [that accounted for nesting]. It was quick, but error prone.
Done by evaluating Character by Character through switches. Here was the basic ...
- 3,545
-1
votes
1
answer
200
views
How can I efficiently test that a parser handles multiple levels of operator precedence correctly?
I'm working on a parser for a (very small) toy language, and I want to test that it's parsing expressions with the appropriate precedence. Previously I just had arithmetic operators, so there weren't ...
- 245
2
votes
1
answer
3k
views
How to parse a dynamically changing Json file? (c#)
So I know a little bit about parsing Json data but not too much so pardon if I am not describing everything as I should. Lets use this Json file as an example:
{
"firstname": "John",
"...
- 21
1
vote
2
answers
280
views
How to concat lists with logical ANDs and ORs
Having multiple lists of integers, like e.g.:
var p1 = new[] { 3, 9, 5, 8, 9 };
var p2 = new[] { 12, 1, 18, 27, 103, 99, 4 };
var p3 = new[] { 23, 930, 15 };
// ...
I want to concatenate them with ...
- 347
1
vote
1
answer
75
views
Lexicon for syntax patterns? [closed]
I am having trouble finding a lexicon which provides terminology for the explicit patterns that are employed when parsing syntax. I am trying to write about the niggling differences between the 10+ ...
- 129
-4
votes
1
answer
280
views
Is it a good idea to use a Parser Combinator to parse unstructured input?
I'm writing a parser that needs to accept unstructured input. By that I mean it needs to take in a raw signal (text, in this case) and look for significant character sequences while accumulating the ...
- 159
0
votes
1
answer
66
views
Parsing tree with many node types
I have a tree data structure where each node describes how to use its child nodes, and this tree is stored in a standard data format - XML for now. However, there are a large number of different kinds ...
- 113
0
votes
1
answer
252
views
Are parser generators useful for parsing a shell language?
From my understanding, parser generators accept as input some form of context-free grammar description. The context-sensitive features are handled during semantic rather than syntactic analysis (...
- 33
5
votes
2
answers
299
views
How to keep parser code and grammar definition in sync?
I am working with a custom, fairly simple DSL for specifying how various scripts will be run. The DSL takes the form of config files that are very simple and easy for humans to read. They define what ...
- 165
0
votes
1
answer
315
views
Object oriented parsing: Is there a pattern or is my approach wrong?
The problem that I am chewing on comes from parsing, i.e. constructing objects in a sequential manner. The grammar is not prefix free, that is, there are more than one syntactical elements sharing the ...
- 367
3
votes
3
answers
1k
views
C++ Recursive Descent Parser: Global Variable Dilemma
I'll come straight to the point. I'm trying to create a Recursive Descent Parser in C++ for a hobby project which involves creating my own minimalist programming language.
One thing that puzzles me, ...
- 41
0
votes
0
answers
424
views
Parsing custom if statement input in PHP
I'm working on a feature where users can get data based on the if statement they write. The if statement looks something like the excel's conditionals.
Basic syntax:
IF ( lhs == rhs, ifTrue, ifFalse)...
- 151
0
votes
1
answer
72
views
Interruptible parsers in Javascript
I’m trying to write a parser in JavaScript that is able to be interrupted by the fact that the entire input source is not available during the parse. When subsequent chunks of the input become ...
- 139
0
votes
0
answers
49
views
Which components of an HTML Element, can I assume will be static over the course of its lifetime?
This is a question for people who are familiar with how HTML typically is built and behaves on webpages.
Backstory and requirements
I am building an HTML tracker with a C++/Qt backend. I am trying ...
- 3,545
2
votes
3
answers
548
views
Is a literal out of range a syntax error or a semantic error?
I am reading more about the differences between syntax and semantics, but I am still wondering about this one.
Let's assume that we have a language that only allows integers to be in the range of 0-...
- 31
0
votes
3
answers
111
views
Options for parsing input that will include math
I'm working on a program that takes in user input which often includes things like math.sqrt(), imaginary numbers, multiplication, division, and similar (basically, standard math plus imaginary ...
- 1,637
1
vote
1
answer
196
views
Question about Backus-NaurForm (BNF)
To write the grammar for Whole Numbers (0,1,2...) in BNF, we may write:
Number ::⇒ Digit MoreDigits
MoreDigits ::⇒
MoreDigits ::⇒ Number
Digit ::⇒ 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9
For a ...
1
vote
2
answers
961
views
Is JavaScript added and executed in parsing or in rendering?
As far as I know, each webpage is created in a two stage process, initiated by a webserver request and ended in a webserver response:
Parsing: markup (Say HTML) is executed as is, or created by ...
user216836
2
votes
1
answer
6k
views
Is it wrong to parse YAML `true` as a string?
Given these lines of YAML:
version: 1.00
y: 1
What does this represent?
According to the YAML spec (I'm not a delicate enough to read the spec like a lawyer), does this necessarily represent that a ...
- 638
1
vote
3
answers
670
views
Generating an AST directly vs. converting from a CST
As I understand it, some parsers generate an abstract syntax tree on the fly, while others first generate a concrete syntax tree and then convert it. What are the tradeoffs between the two? Is there ...
- 153
7
votes
2
answers
1k
views
How can one interpret an Abstract Syntax Tree without recursion?
I am busy building an interpreter for a Rebol-like language. I have previously built a Lisp interpreter where I used recursion, but now I just want to use a stack and not rely on the recursion ...
- 373
1
vote
3
answers
561
views
How to parse DSL file to HTML?
There's DSL format for creating and distributing dictionaries. Every dictionary article in such formats looks like this:
algorithm
[m0][b]al·go·rithm[/b] {{id=000001018}} [c rosybrown]\[[/c][c ...
- 258
0
votes
4
answers
2k
views
Should my lexer allow what is obviously a syntax error?
This is kinda like a concrete version of the question Coming up with tokens for a lexer.
I'm writing a lexer for a small subset of HTML. I'm wondering what should I do when the input stream ends and ...
- 151
-5
votes
1
answer
716
views
LL(K) grammer generation for postfix expressions [closed]
I have a problem in formulating LL(K) grammar for this postfix expression problem, given (4 3 / 2 * 4 5 / +) as an input must output 52/12
1
vote
1
answer
730
views
Rendering custom markdown format: client side or server side?
I am working on a project on both iOS/Android/macOS/Linux (Windows is optional, and surely no website version). There are a lot of related webpage-like data and the team decided to store the data in a ...
- 151
0
votes
1
answer
231
views
Differences between enumeration-based and hierarchical token typing
When writing a lexer/parser, why/when would an advised developer chose to define the tokens' types through an enumeration field/type hierarchy?
The closest question I've found here so far was Lexing: ...
user276191
1
vote
2
answers
8k
views
Should a SVG be converted to JSON to be stored in the database?
Consider an application where you can create and edit a SVG file (e.g. draw.io).
How should the SVG be stored in the database?
With SVG (image/svg+xml)
Thus no further processing is required before ...
- 149