Questions tagged [parsing]

Analyzing (un)structured data to convert it into a structured, normalized format.

Filter by
Sorted by
Tagged with
0 votes
0 answers
49 views

Cloud solution to parse and process 1M+ rows

We are writing a service that can parse a CSV file containing over 1M+ rows. Each row needs insertion (or update) data of a DB record. We are currently using DynamoDB. What AWS services are best ...
user avatar
0 votes
0 answers
50 views

What is it about kdb/q that makes the grammar not suitable for ANTLR style parser generators?

I want to build a code analysis tool for personal use when programming in kdb/q. In order to do this, I need to be able to parse q code into an AST. I have never written a parser before. ANTLR4 seems ...
user avatar
1 vote
2 answers
86 views

What do you call a process which transforms objects of complex types into simple objects of primitive types? [closed]

My first thought was that I'm "serializing" the complex object, but from what I understand that means I'm reducing it down to a string or binary format which could be passed over a network. ...
user avatar
0 votes
1 answer
70 views

Parsing complex data type names

How do compilers parse complex data types names like function pointers. The type has to be somehow put into the AST or it has to be processed during parsing. What are the pros and cons of different ...
user avatar
-1 votes
2 answers
112 views

What would be the performant approach to parsing a _limited subset_ of Markdown? [closed]

There are plenty of articles about parsing full Markdown/CommonMark/etc, but I'm interested only in a specific limited subset, so I'm wondering if a better approach is available to me, and would much ...
user avatar
-1 votes
1 answer
123 views

How does a lexer handle template strings?

So lexers are supposed to emit tokens for key structures like INDENT and DEDENT for indentation stuff, or these: NUMBER ::= [0-9]+ ID ::= [a-Z]+, except for keywords IF ::= 'if' LPAREN ::= '(' ...
user avatar
  • 2,412
0 votes
3 answers
313 views

Is it possible to make a compiler for any dynamic/script/interpreter language

Logically, not based on how cost we will spend or how much we will hire programmers to do it. Can we (Is it possible to) make a compiler for any dynamic/script/interpreter language, like Lua, Python, ...
user avatar
8 votes
1 answer
348 views

What type of syntax notation is this?

SQL Server documentation uses this notation, which is very easy to understand and consume. Is this a BNF Syntax Diagram? Or is this a different type of notation? Source: SQL server documentation page ...
user avatar
1 vote
1 answer
377 views

How does the GLL parsing algorithm work?

I'm very interested in the topic of parsers, especially in the topic of parser combinators like Superpower. The problem with them is that the grammars that they can work with are a bit limited. For ...
user avatar
  • 383
0 votes
1 answer
106 views

Benefits of using a tokenizer/lexer before parsing for recusive descent parser

I am trying to build a static program analyzer for a proprietary progamming language for a school project, and am currently trying to implement the parser from scratch. I was wondering, what are the ...
user avatar
  • 101
15 votes
3 answers
3k views

How would you test a lexer?

I'm wondering how to effectively test a lexer (tokenizer). The number of combinations of tokens in a source file can be huge, and the only way I've found is to make a batch of representative source ...
user avatar
  • 383
1 vote
0 answers
61 views

Is there a easy and useful error handling algorithm for bottom-up based parser?

My English skill is poor because I'm not a native English speaker. Please understand. I wonder that there is a error handling algorithm easy and useful in LR parser. LR Parser is bottom up based so it ...
user avatar
  • 197
6 votes
2 answers
2k views

What are the 'practical' advantages of LR parser over LL parser 'in today'?

My English skill is poor because I'm not a native English speaker. Please understand. I write this article because want to discuss about this topic. I think LR parser has no 'practical' advantages ...
user avatar
  • 197
5 votes
5 answers
472 views

Reasons to use (and not to use) a repeated delimiter to escape that delimiter?

For the designer of a language syntax, what are some reasons to choose a repeated delimiter to escape that delimiter, instead of having a separate escape character to escape that delimiter. A common ...
user avatar
2 votes
1 answer
112 views

Ask for suggestion: data type for parsing stringified fractional numbers

I am the author of a C library for parsing INI files. So far I have delegated the task of parsing values as numbers to the standard atoi() family of functions. However I think time has come that I ...
user avatar
  • 129
-3 votes
1 answer
181 views

Data structures or coding styles in C++ for avoiding long elseif chain when parsing?

Lately I have created some small parsers of data. My initial code structure // more cases here ... else if(!strcmp(X,"somekey")){ // Parse according to "somekey" behavior. } // ...
user avatar
1 vote
3 answers
158 views

Why are code readability and debugging arguments often expressed as a counter-argument for the use of generated LR parsers?

When it comes to using an LR parser generated by a tool, such as Bison, a disadvantage that often comes up as counterarguments is that the resulting parser will be unreadable and complicated to debug, ...
user avatar
  • 219
0 votes
0 answers
170 views

How can you multithread an html parser [ in C++ and similar languages ]?

Ive done two HTML parsers. Done with Regular Expressions [that accounted for nesting]. It was quick, but error prone. Done by evaluating Character by Character through switches. Here was the basic ...
user avatar
  • 3,319
-1 votes
1 answer
149 views

How can I efficiently test that a parser handles multiple levels of operator precedence correctly?

I'm working on a parser for a (very small) toy language, and I want to test that it's parsing expressions with the appropriate precedence. Previously I just had arithmetic operators, so there weren't ...
user avatar
  • 225
2 votes
1 answer
2k views

How to parse a dynamically changing Json file? (c#)

So I know a little bit about parsing Json data but not too much so pardon if I am not describing everything as I should. Lets use this Json file as an example: { "firstname": "John", "...
user avatar
1 vote
2 answers
202 views

How to concat lists with logical ANDs and ORs

Having multiple lists of integers, like e.g.: var p1 = new[] { 3, 9, 5, 8, 9 }; var p2 = new[] { 12, 1, 18, 27, 103, 99, 4 }; var p3 = new[] { 23, 930, 15 }; // ... I want to concatenate them with ...
user avatar
  • 347
1 vote
1 answer
70 views

Lexicon for syntax patterns? [closed]

I am having trouble finding a lexicon which provides terminology for the explicit patterns that are employed when parsing syntax. I am trying to write about the niggling differences between the 10+ ...
user avatar
-4 votes
1 answer
214 views

Is it a good idea to use a Parser Combinator to parse unstructured input?

I'm writing a parser that needs to accept unstructured input. By that I mean it needs to take in a raw signal (text, in this case) and look for significant character sequences while accumulating the ...
user avatar
0 votes
1 answer
64 views

Parsing tree with many node types

I have a tree data structure where each node describes how to use its child nodes, and this tree is stored in a standard data format - XML for now. However, there are a large number of different kinds ...
user avatar
0 votes
1 answer
188 views

Are parser generators useful for parsing a shell language?

From my understanding, parser generators accept as input some form of context-free grammar description. The context-sensitive features are handled during semantic rather than syntactic analysis (...
user avatar
5 votes
2 answers
269 views

How to keep parser code and grammar definition in sync?

I am working with a custom, fairly simple DSL for specifying how various scripts will be run. The DSL takes the form of config files that are very simple and easy for humans to read. They define what ...
user avatar
0 votes
1 answer
229 views

Object oriented parsing: Is there a pattern or is my approach wrong?

The problem that I am chewing on comes from parsing, i.e. constructing objects in a sequential manner. The grammar is not prefix free, that is, there are more than one syntactical elements sharing the ...
user avatar
3 votes
3 answers
1k views

C++ Recursive Descent Parser: Global Variable Dilemma

I'll come straight to the point. I'm trying to create a Recursive Descent Parser in C++ for a hobby project which involves creating my own minimalist programming language. One thing that puzzles me, ...
user avatar
  • 41
0 votes
0 answers
326 views

Parsing custom if statement input in PHP

I'm working on a feature where users can get data based on the if statement they write. The if statement looks something like the excel's conditionals. Basic syntax: IF ( lhs == rhs, ifTrue, ifFalse)...
user avatar
0 votes
1 answer
69 views

Interruptible parsers in Javascript

I’m trying to write a parser in JavaScript that is able to be interrupted by the fact that the entire input source is not available during the parse. When subsequent chunks of the input become ...
user avatar
  • 139
0 votes
0 answers
47 views

Which components of an HTML Element, can I assume will be static over the course of its lifetime?

This is a question for people who are familiar with how HTML typically is built and behaves on webpages. Backstory and requirements I am building an HTML tracker with a C++/Qt backend. I am trying ...
user avatar
  • 3,319
2 votes
3 answers
457 views

Is a literal out of range a syntax error or a semantic error?

I am reading more about the differences between syntax and semantics, but I am still wondering about this one. Let's assume that we have a language that only allows integers to be in the range of 0-...
user avatar
0 votes
3 answers
109 views

Options for parsing input that will include math

I'm working on a program that takes in user input which often includes things like math.sqrt(), imaginary numbers, multiplication, division, and similar (basically, standard math plus imaginary ...
user avatar
  • 1,637
1 vote
1 answer
165 views

Question about Backus-NaurForm (BNF)

To write the grammar for Whole Numbers (0,1,2...) in BNF, we may write: Number ::⇒ Digit MoreDigits MoreDigits ::⇒ MoreDigits ::⇒ Number Digit ::⇒ 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 For a ...
user avatar
1 vote
2 answers
798 views

Is JavaScript added and executed in parsing or in rendering?

As far as I know, each webpage is created in a two stage process, initiated by a webserver request and ended in a webserver response: Parsing: markup (Say HTML) is executed as is, or created by ...
user avatar
2 votes
1 answer
4k views

Is it wrong to parse YAML `true` as a string?

Given these lines of YAML: version: 1.00 y: 1 What does this represent? According to the YAML spec (I'm not a delicate enough to read the spec like a lawyer), does this necessarily represent that a ...
user avatar
  • 618
1 vote
3 answers
449 views

Generating an AST directly vs. converting from a CST

As I understand it, some parsers generate an abstract syntax tree on the fly, while others first generate a concrete syntax tree and then convert it. What are the tradeoffs between the two? Is there ...
user avatar
  • 153
6 votes
2 answers
1k views

How can one interpret an Abstract Syntax Tree without recursion?

I am busy building an interpreter for a Rebol-like language. I have previously built a Lisp interpreter where I used recursion, but now I just want to use a stack and not rely on the recursion ...
user avatar
1 vote
3 answers
419 views

How to parse DSL file to HTML?

There's DSL format for creating and distributing dictionaries. Every dictionary article in such formats looks like this: algorithm [m0][b]al·go·rithm[/b] {{id=000001018}} [c rosybrown]\[[/c][c ...
user avatar
  • 208
0 votes
4 answers
1k views

Should my lexer allow what is obviously a syntax error?

This is kinda like a concrete version of the question Coming up with tokens for a lexer. I'm writing a lexer for a small subset of HTML. I'm wondering what should I do when the input stream ends and ...
user avatar
-5 votes
1 answer
474 views

LL(K) grammer generation for postfix expressions [closed]

I have a problem in formulating LL(K) grammar for this postfix expression problem, given (4 3 / 2 * 4 5 / +) as an input must output 52/12
user avatar
1 vote
1 answer
625 views

Rendering custom markdown format: client side or server side?

I am working on a project on both iOS/Android/macOS/Linux (Windows is optional, and surely no website version). There are a lot of related webpage-like data and the team decided to store the data in a ...
user avatar
0 votes
1 answer
177 views

Differences between enumeration-based and hierarchical token typing

When writing a lexer/parser, why/when would an advised developer chose to define the tokens' types through an enumeration field/type hierarchy? The closest question I've found here so far was Lexing: ...
user avatar
1 vote
2 answers
7k views

Should a SVG be converted to JSON to be stored in the database?

Consider an application where you can create and edit a SVG file (e.g. draw.io). How should the SVG be stored in the database? With SVG (image/svg+xml) Thus no further processing is required before ...
user avatar
0 votes
2 answers
3k views

Understanding Context Free Grammar using a simple C code

I'm trying to understand the difference between terminal and non-terminal values in a real language. I wasn't able to find enough examples on real language CFGs on the internet, most examples are ...
user avatar
  • 11
2 votes
3 answers
865 views

Catch statements with other Business Logic in Parsing Files

I am writing a simple program to parse a text file and place into Generic List. Sample Text: 1,Joe,CA,58,2 2,Matt,TX,63,5 Sometimes, there may be an error, with missing data in the file 1,Joe,CA,...
user avatar
0 votes
2 answers
226 views

Is there a name for these diagrams that show the flow of a valid value [duplicate]

Here's an example taken from http://json.org/ Is there a name for this kind of diagram?
user avatar
  • 476
2 votes
0 answers
281 views

What obstacles should I expect in building universal Abstract Syntax Tree's manipulation tool?

I am trying to build code generation and transformation tool and hence I need software for Abstract Syntax Tree manipulation for at least three languages (Coq, Haskell, Java). So far I have managed to ...
user avatar
  • 1,001
1 vote
1 answer
701 views

Parsing huge XML files

our startup is working on an integration for various customers. All of the customers are required to provide their data in one unified way = we chose XML files because of legacy compatibility and ease ...
user avatar
-2 votes
3 answers
198 views

Parsing Complex Context-Sensitive Text

I am trying to learn how to handle complex context-sensitivity in parsing. Say you have this simple example of code: var x = 10 var y = x + 10 var z = y + 10 var a = z + 10 var b = a + 10 var c = b + ...
user avatar
  • 2,412

1
2 3 4 5 6