Questions tagged [parsing]

Analyzing (un)structured data to convert it into a structured, normalized format.

Filter by
Sorted by
Tagged with
0
votes
1answer
93 views

Are parser generators useful for parsing a shell language?

From my understanding, parser generators accept as input some form of context-free grammar description. The context-sensitive features are handled during semantic rather than syntactic analysis (...
4
votes
2answers
146 views

How to keep parser code and grammar definition in sync?

I am working with a custom, fairly simple DSL for specifying how various scripts will be run. The DSL takes the form of config files that are very simple and easy for humans to read. They define what ...
0
votes
0answers
71 views

Parsing custom if statement input in PHP

I'm working on a feature where users can get data based on the if statement they write. The if statement looks something like the excel's conditionals. Basic syntax: IF ( lhs == rhs, ifTrue, ifFalse)...
3
votes
3answers
299 views

C++ Recursive Descent Parser: Global Variable Dilemma

I'll come straight to the point. I'm trying to create a Recursive Descent Parser in C++ for a hobby project which involves creating my own minimalist programming language. One thing that puzzles me, ...
0
votes
1answer
132 views

Object oriented parsing: Is there a pattern or is my approach wrong?

The problem that I am chewing on comes from parsing, i.e. constructing objects in a sequential manner. The grammar is not prefix free, that is, there are more than one syntactical elements sharing the ...
0
votes
1answer
51 views

Interruptible parsers in Javascript

I’m trying to write a parser in JavaScript that is able to be interrupted by the fact that the entire input source is not available during the parse. When subsequent chunks of the input become ...
0
votes
1answer
85 views

Dedupe while or after write

I have a summary tool written in python that reads input files, and writes them into a summary file. I have the following stipulations: No duplicates. If it exists, add a count to it. Is it better ...
0
votes
0answers
36 views

Which components of an HTML Element, can I assume will be static over the course of its lifetime?

This is a question for people who are familiar with how HTML typically is built and behaves on webpages. Backstory and requirements I am building an HTML tracker with a C++/Qt backend. I am trying ...
2
votes
3answers
235 views

Is a literal out of range a syntax error or a semantic error?

I am reading more about the differences between syntax and semantics, but I am still wondering about this one. Let's assume that we have a language that only allows integers to be in the range of 0-...
4
votes
2answers
457 views

Correct terminology for if-blocks

I am using lex/yacc to parse a simple language which allows 'if blocks': IF something something [ELSE IF something][ELSE] END IF and also other nested logic under a 'DEFINE' statement: DEFINE ...
0
votes
3answers
100 views

Options for parsing input that will include math

I'm working on a program that takes in user input which often includes things like math.sqrt(), imaginary numbers, multiplication, division, and similar (basically, standard math plus imaginary ...
0
votes
1answer
69 views

Differences between enumeration-based and hierarchical token typing

When writing a lexer/parser, why/when would an advised developer chose to define the tokens' types through an enumeration field/type hierarchy? The closest question I've found here so far was Lexing: ...
8
votes
1answer
986 views

quantitatively comparing AST shapes

How could one compare the shape of abstract syntax trees of similar source code programs (C, C++, Go, or anything compiled with GCC...)? I guess that plagiarism detection on source code would use ...
1
vote
1answer
104 views

Question about Backus-NaurForm (BNF)

To write the grammar for Whole Numbers (0,1,2...) in BNF, we may write: Number ::⇒ Digit MoreDigits MoreDigits ::⇒ MoreDigits ::⇒ Number Digit ::⇒ 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 For a ...
0
votes
1answer
127 views

Is JavaScript added and executed in rendering or parsing?

AFAIK, each web page is created in a two stage process: Parsing and rendering. In parsing, markup (Say HTML) is taken as is, or created from a server-side programming language like PHP, In the ...
1
vote
1answer
391 views

Desired pattern for expression parsing into RPN

I'm writing a interpreter/compiler I have encountered a problem which I have solved before, but not optimally perhaps. The problem goes like this, expressions can express a lot of things like scopes,...
1
vote
1answer
153 views

Is it wrong to parse YAML `true` as a string?

Given these lines of YAML: version: 1.00 y: 1 What does this represent? According to the YAML spec (I'm not a delicate enough to read the spec like a lawyer), does this necessarily represent that a ...
0
votes
2answers
95 views

Generating an AST directly vs. converting from a CST

As I understand it, some parsers generate an abstract syntax tree on the fly, while others first generate a concrete syntax tree and then convert it. What are the tradeoffs between the two? Is there ...
4
votes
2answers
296 views

How can one interpret an Abstract Syntax Tree without recursion?

I am busy building an interpreter for a Rebol-like language. I have previously built a Lisp interpreter where I used recursion, but now I just want to use a stack and not rely on the recursion ...
1
vote
2answers
143 views

How to parse DSL file to HTML?

There's DSL format for creating and distributing dictionaries. Every dictionary article in such formats looks like this: algorithm [m0][b]al·go·rithm[/b] {{id=000001018}} [c rosybrown]\[[/c][c ...
14
votes
2answers
6k views

Do compilers utilize multithreading for faster compile times?

If I remember my compilers course correctly, the typical compiler has the following simplified outline: A lexical analyzer scans (or calls some scanning function on) the source code character-by-...
0
votes
4answers
259 views

Should my lexer allow what is obviously a syntax error?

This is kinda like a concrete version of the question Coming up with tokens for a lexer. I'm writing a lexer for a small subset of HTML. I'm wondering what should I do when the input stream ends and ...
-5
votes
1answer
83 views

LL(K) grammer generation for postfix expressions [closed]

I have a problem in formulating LL(K) grammar for this postfix expression problem, given (4 3 / 2 * 4 5 / +) as an input must output 52/12
1
vote
1answer
225 views

Rendering custom markdown format: client side or server side?

I am working on a project on both iOS/Android/macOS/Linux (Windows is optional, and surely no website version). There are a lot of related webpage-like data and the team decided to store the data in a ...
0
votes
2answers
2k views

Should a SVG be converted to JSON to be stored in the database?

Consider an application where you can create and edit a SVG file (e.g. draw.io). How should the SVG be stored in the database? With SVG (image/svg+xml) Thus no further processing is required before ...
0
votes
2answers
663 views

Understanding Context Free Grammar using a simple C code

I'm trying to understand the difference between terminal and non-terminal values in a real language. I wasn't able to find enough examples on real language CFGs on the internet, most examples are ...
1
vote
3answers
200 views

Catch statements with other Business Logic in Parsing Files

I am writing a simple program to parse a text file and place into Generic List. Sample Text: 1,Joe,CA,58,2 2,Matt,TX,63,5 Sometimes, there may be an error, with missing data in the file 1,Joe,CA,...
0
votes
2answers
157 views

Is there a name for these diagrams that show the flow of a valid value [duplicate]

Here's an example taken from http://json.org/ Is there a name for this kind of diagram?
58
votes
4answers
9k views

When to use a Parser Combinator? When to use a Parser Generator?

I've taken a deep dive into the world of parsers recently, wanting to create my own programming language. However, I found out that there exist two somewhat different approaches of writing parsers: ...
1
vote
0answers
100 views

What obstacles should I expect in building universal Abstract Syntax Tree's manipulation tool?

I am trying to build code generation and transformation tool and hence I need software for Abstract Syntax Tree manipulation for at least three languages (Coq, Haskell, Java). So far I have managed to ...
1
vote
1answer
221 views

Parsing huge XML files

our startup is working on an integration for various customers. All of the customers are required to provide their data in one unified way = we chose XML files because of legacy compatibility and ease ...
-2
votes
3answers
152 views

Parsing Complex Context-Sensitive Text

I am trying to learn how to handle complex context-sensitivity in parsing. Say you have this simple example of code: var x = 10 var y = x + 10 var z = y + 10 var a = z + 10 var b = a + 10 var c = b + ...
1
vote
1answer
218 views

Matching groups of similar lines on a generic matching algorithm

I have to write a program to search through a file containing lines and find lines that match to a degree of tolerance but are not necessarily the same. So for example the following lines would match: ...
0
votes
1answer
211 views

What algorithm to use to parse expressions in this simple language?

I don't want to include the entirety of my language, but I think the relevant part is: <expr> ::= <logical> | <comparison> | <constant> | <...
0
votes
1answer
251 views

Do any programming languages let you use other languages without restriction within them?

This may be a stupid question, and it would certainly take one Hell of a lexer, but do any extant programming languages allow you to do something like: c# (1.2) { // c# code } Perl (5) { # ...
2
votes
1answer
172 views

One data structure to both parse and stringify

You can create a data structure such as a Parsing Expression Grammar (PEG) that will be used to parse: string -> object You can then write a function that iterates through the object's properties ...
1
vote
1answer
87 views

Backend - own database vs live parsing

I am developing a mobile app that depends on some nearly static data. I will gather that data initially from an existing website by doing some HTML parsing. I could now create a database from that ...
3
votes
1answer
448 views

Is it worth parsing an infix algebraic expression to postfix and then an expression tree?

I am trying to make a simple expression parser, in which users type an expression in infix notation (for example ln((1+2)(3-4))) to apply some calculations. To make this possible I need to tokenize ...
0
votes
1answer
362 views

AST design: Call is both expression and statement?

I'm designing the AST for a compiler and found that I don't actually know how to represent the Call node. Currently, the relevant part of the AST looks like this (where the arrows denote inheritance):...
1
vote
4answers
661 views

Are parsers a special case in unit testing?

Context While trying to get my parser classes under test, I noticed a common challenge for (unit-)testing them: They have only one public method, with a string as input and the parsed class as output....
3
votes
0answers
44 views

Parse track meet output file result

I am building out a parser for a well-known track & field meet management software output file (sample output). I can parse this manually, and put items into my object model (overview below), but ...
47
votes
4answers
58k views

How exactly is an Abstract Syntax Tree created?

I think I understand the goal of an AST, and I've built a couple of tree structures before, but never an AST. I'm mostly confused because the nodes are text and not number, so I can't think of a nice ...
19
votes
5answers
30k views

Can the csv format be defined by a regex?

A colleague and I have recently argued over whether a pure regex is capable of fully encapsulating the csv format, such that it is capable of parsing all files with any given escape char, quote char, ...
7
votes
5answers
2k views

How to Unit-Test a parser of a file?

I'm implementing a metadata parser of image files from all formats. I want to write tests for it. One trivial way to do so is to have test image files of all formats as a resources for the tests, and ...
4
votes
2answers
518 views

Lexer/Parser for multidimensional Languages

How does Lexer/Parser work in a 2D programming language like Funciton in order to transform such an unusual source-code to the correct AST?
2
votes
2answers
123 views

Designing for the validation of phone numbers

Let's say we have a registration system that finalises the creation of an account by SMSing the user their new account details. This system could be used by people all around the world, so let's say ...
1
vote
1answer
179 views

Can I get information about an SQL SELECT query (such as the tables it accesses) using C#?

The c# application I'm working with executes many T-SQL SELECT queries that are stored as strings in a database. These queries almost all involve selecting data from and joining with several tables ...
0
votes
1answer
397 views

Grammar by example

Is there an automated approach to generate a grammar (which could be used later in a compiler tool such as ANTLR or similar) from given examples of a language? With more detail: assumed a technical ...
3
votes
1answer
82 views

What approaches can I take to figure out the “relevancy” of certain terms in a string?

I'm not even sure "relevancy" is the most accurate word, so I'll just describe the problem: I'm building an app that needs to somehow parse product descriptions from a popular website (let's just say ...
9
votes
1answer
471 views

What makes some things easier to parse than others?

I was just reading the Wikipedia page for WebAssembly and it says: “WebAssembly is… designed to be faster to parse than JavaScript” , which got me thinking, what makes a certain language or data ...