Questions tagged [parsing]

Analyzing (un)structured data to convert it into a structured, normalized format.

2
votes
3answers
200 views

Is a literal out of range a syntax error or a semantic error?

I am reading more about the differences between syntax and semantics, but I am still wondering about this one. Let's assume that we have a language that only allows integers to be in the range of 0-...
0
votes
3answers
97 views

Options for parsing input that will include math

I'm working on a program that takes in user input which often includes things like math.sqrt(), imaginary numbers, multiplication, division, and similar (basically, standard math plus imaginary ...
1
vote
1answer
93 views

Question about Backus-NaurForm (BNF)

To write the grammar for Whole Numbers (0,1,2...) in BNF, we may write: Number ::⇒ Digit MoreDigits MoreDigits ::⇒ MoreDigits ::⇒ Number Digit ::⇒ 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 For a ...
0
votes
1answer
80 views

Is JavaScript added and executed in rendering or parsing?

AFAIK, each web page is created in a two stage process: Parsing and rendering. In parsing, markup (Say HTML) is taken as is, or created from a server-side programming language like PHP, In the ...
1
vote
1answer
145 views

Is it wrong to parse YAML `true` as a string?

Given these lines of YAML: version: 1.00 y: 1 What does this represent? According to the YAML spec (I'm not a delicate enough to read the spec like a lawyer), does this necessarily represent that a ...
0
votes
2answers
88 views

Generating an AST directly vs. converting from a CST

As I understand it, some parsers generate an abstract syntax tree on the fly, while others first generate a concrete syntax tree and then convert it. What are the tradeoffs between the two? Is there ...
4
votes
2answers
242 views

How can one interpret an Abstract Syntax Tree without recursion?

I am busy building an interpreter for a Rebol-like language. I have previously built a Lisp interpreter where I used recursion, but now I just want to use a stack and not rely on the recursion ...
1
vote
2answers
126 views

How to parse DSL file to HTML?

There's DSL format for creating and distributing dictionaries. Every dictionary article in such formats looks like this: algorithm [m0][b]al·go·rithm[/b] {{id=000001018}} [c rosybrown]\[[/c][c ...
0
votes
4answers
184 views

Should my lexer allow what is obviously a syntax error?

This is kinda like a concrete version of the question Coming up with tokens for a lexer. I'm writing a lexer for a small subset of HTML. I'm wondering what should I do when the input stream ends and ...
-5
votes
1answer
69 views

LL(K) grammer generation for postfix expressions [closed]

I have a problem in formulating LL(K) grammar for this postfix expression problem, given (4 3 / 2 * 4 5 / +) as an input must output 52/12
1
vote
1answer
175 views

Rendering custom markdown format: client side or server side?

I am working on a project on both iOS/Android/macOS/Linux (Windows is optional, and surely no website version). There are a lot of related webpage-like data and the team decided to store the data in a ...
0
votes
1answer
57 views

Differences between enumeration-based and hierarchical token typing

When writing a lexer/parser, why/when would an advised developer chose to define the tokens' types through an enumeration field/type hierarchy? The closest question I've found here so far was Lexing: ...
0
votes
2answers
955 views

Should a SVG be converted to JSON to be stored in the database?

Consider an application where you can create and edit a SVG file (e.g. draw.io). How should the SVG be stored in the database? With SVG (image/svg+xml) Thus no further processing is required before ...
0
votes
2answers
457 views

Understanding Context Free Grammar using a simple C code

I'm trying to understand the difference between terminal and non-terminal values in a real language. I wasn't able to find enough examples on real language CFGs on the internet, most examples are ...
1
vote
3answers
121 views

Catch statements with other Business Logic in Parsing Files

I am writing a simple program to parse a text file and place into Generic List. Sample Text: 1,Joe,CA,58,2 2,Matt,TX,63,5 Sometimes, there may be an error, with missing data in the file 1,Joe,CA,...
0
votes
2answers
138 views

Is there a name for these diagrams that show the flow of a valid value [duplicate]

Here's an example taken from http://json.org/ Is there a name for this kind of diagram?
0
votes
0answers
13 views

General structures / algorithms for parsing / interpreting a semantic query efficiently

If you have a semantic/meaning graph data structure built up, and you get a new word as part of a sentence, the question is, how it optimally determines the meaning of that new word. It seems that, ...
1
vote
0answers
87 views

What obstacles should I expect in building universal Abstract Syntax Tree's manipulation tool?

I am trying to build code generation and transformation tool and hence I need software for Abstract Syntax Tree manipulation for at least three languages (Coq, Haskell, Java). So far I have managed to ...
1
vote
1answer
184 views

Parsing huge XML files

our startup is working on an integration for various customers. All of the customers are required to provide their data in one unified way = we chose XML files because of legacy compatibility and ease ...
-2
votes
3answers
143 views

Parsing Complex Context-Sensitive Text

I am trying to learn how to handle complex context-sensitivity in parsing. Say you have this simple example of code: var x = 10 var y = x + 10 var z = y + 10 var a = z + 10 var b = a + 10 var c = b + ...
0
votes
1answer
165 views

What algorithm to use to parse expressions in this simple language?

I don't want to include the entirety of my language, but I think the relevant part is: <expr> ::= <logical> | <comparison> | <constant> | <...
0
votes
1answer
244 views

Do any programming languages let you use other languages without restriction within them?

This may be a stupid question, and it would certainly take one Hell of a lexer, but do any extant programming languages allow you to do something like: c# (1.2) { // c# code } Perl (5) { # ...
2
votes
1answer
162 views

One data structure to both parse and stringify

You can create a data structure such as a Parsing Expression Grammar (PEG) that will be used to parse: string -> object You can then write a function that iterates through the object's properties ...
1
vote
1answer
82 views

Backend - own database vs live parsing

I am developing a mobile app that depends on some nearly static data. I will gather that data initially from an existing website by doing some HTML parsing. I could now create a database from that ...
0
votes
1answer
320 views

AST design: Call is both expression and statement?

I'm designing the AST for a compiler and found that I don't actually know how to represent the Call node. Currently, the relevant part of the AST looks like this (where the arrows denote inheritance):...
1
vote
4answers
562 views

Are parsers a special case in unit testing?

Context While trying to get my parser classes under test, I noticed a common challenge for (unit-)testing them: They have only one public method, with a string as input and the parsed class as output....
2
votes
0answers
41 views

Parse track meet output file result

I am building out a parser for a well-known track & field meet management software output file (sample output). I can parse this manually, and put items into my object model (overview below), but ...
7
votes
5answers
2k views

How to Unit-Test a parser of a file?

I'm implementing a metadata parser of image files from all formats. I want to write tests for it. One trivial way to do so is to have test image files of all formats as a resources for the tests, and ...
2
votes
2answers
119 views

Designing for the validation of phone numbers

Let's say we have a registration system that finalises the creation of an account by SMSing the user their new account details. This system could be used by people all around the world, so let's say ...
1
vote
1answer
169 views

Can I get information about an SQL SELECT query (such as the tables it accesses) using C#?

The c# application I'm working with executes many T-SQL SELECT queries that are stored as strings in a database. These queries almost all involve selecting data from and joining with several tables ...
0
votes
1answer
379 views

Grammar by example

Is there an automated approach to generate a grammar (which could be used later in a compiler tool such as ANTLR or similar) from given examples of a language? With more detail: assumed a technical ...
3
votes
1answer
82 views

What approaches can I take to figure out the “relevancy” of certain terms in a string?

I'm not even sure "relevancy" is the most accurate word, so I'll just describe the problem: I'm building an app that needs to somehow parse product descriptions from a popular website (let's just say ...
3
votes
1answer
101 views

Meaningful response to the user after his uploaded CSV was processed?

A user uploads a CSV to the server to be inserted into the database and I would like to return a meaningful response to this user. My thoughts : { inserted : x records discarded : y records } ...
0
votes
1answer
100 views

Optimal pattern for parsing delimited (e.g. new line) network stream

I am looking for optimal for maintenance/easy of programming and elegant patterns to parse delimited network streams, e.g. protocols like SMTP. Some of the concerns I've always had are: how to react ...
3
votes
1answer
366 views

Is it worth parsing an infix algebraic expression to postfix and then an expression tree?

I am trying to make a simple expression parser, in which users type an expression in infix notation (for example ln((1+2)(3-4))) to apply some calculations. To make this possible I need to tokenize ...
-2
votes
1answer
119 views

Multiple variable declaration, multiple variable assignment, context-sensitive 'in' statement

Lately I've been playing with writing my own programming language, following the excellent Crafting Interpreters book but I've hit something of a snag. I'd like to extend the parser to accept ...
-1
votes
1answer
140 views

How do different file types generally store data?

I am working on a project, for which I want to create my own file format to store large amounts of data. I am trying to decide how that file format should be built to be as efficient as possible. The ...
3
votes
1answer
533 views

Which is a more efficient approach to decoding escape sequences in text?

I'm working on parsers that not only process delimited content, but also escape sequences within certain portions of that content. I'm contemplating the efficiency of several approaches to ...
0
votes
1answer
246 views

Designing syntax tree derived node classes

Let's say I have an abstract Node class and BinaryOperation class derived from Node and three classes IntegerConstant, RealConstant and CharacterConstant also derived from Node. The general design ...
1
vote
1answer
365 views

Desired pattern for expression parsing into RPN

I'm writing a interpreter/compiler I have encountered a problem which I have solved before, but not optimally perhaps. The problem goes like this, expressions can express a lot of things like scopes,...
2
votes
2answers
261 views

Is implementing operator overloading with variable arity possible?

Implementing operator overloading for e.g. + (plus) isn't terribly difficult if you know it's a binary operator. One can just parse expression + expression But what if the programmer can choose ...
0
votes
3answers
2k views

What approaches can be used to convert existing JSON structure to existing XML structure?

I am working on a component for an enterprise message queue that consumes a message from a queue and sends it as a text message. The message is picked up by my consumer in JSON format (I do not have ...
1
vote
4answers
355 views

Is it common to write File Parsers in higher languages like Java?

I work with an older framework (15 years+) at work which is fully built on Perl. Its main tasks are calling external programs via command-line calls, parsing their output and presenting some of the ...
4
votes
2answers
455 views

Correct terminology for if-blocks

I am using lex/yacc to parse a simple language which allows 'if blocks': IF something something [ELSE IF something][ELSE] END IF and also other nested logic under a 'DEFINE' statement: DEFINE ...
0
votes
0answers
63 views

Parsing nested structure from facebook pfff - how to decide on parser

I'm playing around since a while with building a little open source project based on the output of facebooks pfff tool, that can parse multiple programming languages and output some kind of unified ...
5
votes
3answers
633 views

Proper separation between lexing and parsing

I am currently writing a parser which, given a source file, turns it into an AST of some language, respecting the idiomatic process of lexing and then parsing using well-known parser generators (think ...
1
vote
0answers
94 views

Concurrently parsing records in a binary file in Go

I have a binary file that I want to parse. The file is broken up into records that are 1024 bytes each. The high level steps needed are: Read 1024 bytes at a time from the file. Parse each 1024-byte "...
3
votes
3answers
332 views

How to create simulator for distributed algorithms written in simple language

I started development of simulator for simulation of distributed algorithms in language C. My work consist of creating simple language for algorithm description and simulator which takes the described ...
9
votes
1answer
460 views

What makes some things easier to parse than others?

I was just reading the Wikipedia page for WebAssembly and it says: “WebAssembly is… designed to be faster to parse than JavaScript” , which got me thinking, what makes a certain language or data ...
1
vote
2answers
166 views

Is there a good single byte delimeter for use with utf-8 strings that isn't a null terminator?

I'm looking for a quick way to split strings containing individual JSON payloads. Currently, I'm using newlines and searching for the newline ASCII character, but I figure if I start using utf-8 this ...