Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [parsing]

Analyzing (un)structured data to convert it into a structured, normalized format.

0
votes
4answers
140 views

Should my lexer allow what is obviously a syntax error?

This is kinda like a concrete version of the question Coming up with tokens for a lexer. I'm writing a lexer for a small subset of HTML. I'm wondering what should I do when the input stream ends and ...
-4
votes
1answer
38 views

LL(K) grammer generation for postfix expressions [closed]

I have a problem in formulating LL(K) grammar for this postfix expression problem, given (4 3 / 2 * 4 5 / +) as an input must output 52/12
1
vote
1answer
82 views

Rendering custom markdown format: client side or server side?

I am working on a project on both iOS/Android/macOS/Linux (Windows is optional, and surely no website version). There are a lot of related webpage-like data and the team decided to store the data in a ...
-1
votes
1answer
36 views

Differences between enumeration-based and hierarchical token typing

When writing a lexer/parser, why/when would an advised developer chose to define the tokens' types through an enumeration field/type hierarchy? The closest question I've found here so far was Lexing: ...
0
votes
2answers
118 views

Should a SVG be converted to JSON to be stored in the database?

Consider an application where you can create and edit a SVG file (e.g. draw.io). How should the SVG be stored in the database? With SVG (image/svg+xml) Thus no further processing is required before ...
0
votes
2answers
81 views

Understanding Context Free Grammar using a simple C code

I'm trying to understand the difference between terminal and non-terminal values in a real language. I wasn't able to find enough examples on real language CFGs on the internet, most examples are ...
1
vote
3answers
86 views

Catch statements with other Business Logic in Parsing Files

I am writing a simple program to parse a text file and place into Generic List. Sample Text: 1,Joe,CA,58,2 2,Matt,TX,63,5 Sometimes, there may be an error, with missing data in the file 1,Joe,CA,...
0
votes
2answers
107 views

Is there a name for these diagrams that show the flow of a valid value [duplicate]

Here's an example taken from http://json.org/ Is there a name for this kind of diagram?
0
votes
0answers
18 views

Design of a schema parser in order to validate files according to that schema?

I am trying to parse files written in the EXPRESS data modeling language. These files are used to define data structures of files used for the exchange of product manufacturing data. In my case, this ...
0
votes
0answers
12 views

General structures / algorithms for parsing / interpreting a semantic query efficiently

If you have a semantic/meaning graph data structure built up, and you get a new word as part of a sentence, the question is, how it optimally determines the meaning of that new word. It seems that, ...
1
vote
0answers
70 views

What obstacles should I expect in building universal Abstract Syntax Tree's manipulation tool?

I am trying to build code generation and transformation tool and hence I need software for Abstract Syntax Tree manipulation for at least three languages (Coq, Haskell, Java). So far I have managed to ...
1
vote
1answer
132 views

Parsing huge XML files

our startup is working on an integration for various customers. All of the customers are required to provide their data in one unified way = we chose XML files because of legacy compatibility and ease ...
-2
votes
3answers
113 views

Parsing Complex Context-Sensitive Text

I am trying to learn how to handle complex context-sensitivity in parsing. Say you have this simple example of code: var x = 10 var y = x + 10 var z = y + 10 var a = z + 10 var b = a + 10 var c = b + ...
0
votes
1answer
88 views

What algorithm to use to parse expressions in this simple language?

I don't want to include the entirety of my language, but I think the relevant part is: <expr> ::= <logical> | <comparison> | <constant> | <...
0
votes
0answers
37 views

Is re-using parts of older mapper/parser versions advisable?

Context I'm working on an app that uses the JSON format for persistence on a web server. The app is in production and features are being added over time, so from time to time there's some additional ...
0
votes
1answer
224 views

Do any programming languages let you use other languages without restriction within them?

This may be a stupid question, and it would certainly take one Hell of a lexer, but do any extant programming languages allow you to do something like: c# (1.2) { // c# code } Perl (5) { # ...
2
votes
1answer
151 views

One data structure to both parse and stringify

You can create a data structure such as a Parsing Expression Grammar (PEG) that will be used to parse: string -> object You can then write a function that iterates through the object's properties ...
1
vote
1answer
75 views

Backend - own database vs live parsing

I am developing a mobile app that depends on some nearly static data. I will gather that data initially from an existing website by doing some HTML parsing. I could now create a database from that ...
0
votes
1answer
163 views

AST design: Call is both expression and statement?

I'm designing the AST for a compiler and found that I don't actually know how to represent the Call node. Currently, the relevant part of the AST looks like this (where the arrows denote inheritance):...
1
vote
4answers
327 views

Are parsers a special case in unit testing?

Context While trying to get my parser classes under test, I noticed a common challenge for (unit-)testing them: They have only one public method, with a string as input and the parsed class as output....
3
votes
0answers
34 views

Parse track meet output file result

I am building out a parser for a well-known track & field meet management software output file (sample output). I can parse this manually, and put items into my object model (overview below), but ...
5
votes
5answers
798 views

How to Unit-Test a parser of a file?

I'm implementing a metadata parser of image files from all formats. I want to write tests for it. One trivial way to do so is to have test image files of all formats as a resources for the tests, and ...
2
votes
2answers
111 views

Designing for the validation of phone numbers

Let's say we have a registration system that finalises the creation of an account by SMSing the user their new account details. This system could be used by people all around the world, so let's say ...
1
vote
1answer
148 views

Can I get information about an SQL SELECT query (such as the tables it accesses) using C#?

The c# application I'm working with executes many T-SQL SELECT queries that are stored as strings in a database. These queries almost all involve selecting data from and joining with several tables ...
0
votes
1answer
304 views

Grammar by example

Is there an automated approach to generate a grammar (which could be used later in a compiler tool such as ANTLR or similar) from given examples of a language? With more detail: assumed a technical ...
3
votes
1answer
81 views

What approaches can I take to figure out the “relevancy” of certain terms in a string?

I'm not even sure "relevancy" is the most accurate word, so I'll just describe the problem: I'm building an app that needs to somehow parse product descriptions from a popular website (let's just say ...
3
votes
1answer
98 views

Meaningful response to the user after his uploaded CSV was processed?

A user uploads a CSV to the server to be inserted into the database and I would like to return a meaningful response to this user. My thoughts : { inserted : x records discarded : y records } ...
0
votes
1answer
94 views

Optimal pattern for parsing delimited (e.g. new line) network stream

I am looking for optimal for maintenance/easy of programming and elegant patterns to parse delimited network streams, e.g. protocols like SMTP. Some of the concerns I've always had are: how to react ...
3
votes
1answer
219 views

Is it worth parsing an infix algebraic expression to postfix and then an expression tree?

I am trying to make a simple expression parser, in which users type an expression in infix notation (for example ln((1+2)(3-4))) to apply some calculations. To make this possible I need to tokenize ...
-3
votes
1answer
115 views

Multiple variable declaration, multiple variable assignment, context-sensitive 'in' statement

Lately I've been playing with writing my own programming language, following the excellent Crafting Interpreters book but I've hit something of a snag. I'd like to extend the parser to accept ...
-1
votes
1answer
128 views

How do different file types generally store data?

I am working on a project, for which I want to create my own file format to store large amounts of data. I am trying to decide how that file format should be built to be as efficient as possible. The ...
3
votes
1answer
377 views

Which is a more efficient approach to decoding escape sequences in text?

I'm working on parsers that not only process delimited content, but also escape sequences within certain portions of that content. I'm contemplating the efficiency of several approaches to ...
0
votes
1answer
178 views

Designing syntax tree derived node classes

Let's say I have an abstract Node class and BinaryOperation class derived from Node and three classes IntegerConstant, RealConstant and CharacterConstant also derived from Node. The general design ...
1
vote
1answer
292 views

Desired pattern for expression parsing into RPN

Im writing a interpreter/compiler I have encountered a problem which I have solved before, but not optimally perhaps. The problem goes like this, expressions can express alot of things like scopes, ...
2
votes
2answers
207 views

Is implementing operator overloading with variable arity possible?

Implementing operator overloading for e.g. + (plus) isn't terribly difficult if you know it's a binary operator. One can just parse expression + expression But what if the programmer can choose ...
0
votes
3answers
1k views

What approaches can be used to convert existing JSON structure to existing XML structure?

I am working on a component for an enterprise message queue that consumes a message from a queue and sends it as a text message. The message is picked up by my consumer in JSON format (I do not have ...
1
vote
4answers
352 views

Is it common to write File Parsers in higher languages like Java?

I work with an older framework (15 years+) at work which is fully built on Perl. Its main tasks are calling external programs via command-line calls, parsing their output and presenting some of the ...
4
votes
2answers
313 views

Correct terminology for if-blocks

I am using lex/yacc to parse a simple language which allows 'if blocks': IF something something [ELSE IF something][ELSE] END IF and also other nested logic under a 'DEFINE' statement: DEFINE ...
0
votes
0answers
54 views

Parsing nested structure from facebook pfff - how to decide on parser

I'm playing around since a while with building a little open source project based on the output of facebooks pfff tool, that can parse multiple programming languages and output some kind of unified ...
5
votes
3answers
298 views

Proper separation between lexing and parsing

I am currently writing a parser which, given a source file, turns it into an AST of some language, respecting the idiomatic process of lexing and then parsing using well-known parser generators (think ...
1
vote
0answers
83 views

Concurrently parsing records in a binary file in Go

I have a binary file that I want to parse. The file is broken up into records that are 1024 bytes each. The high level steps needed are: Read 1024 bytes at a time from the file. Parse each 1024-byte "...
4
votes
3answers
270 views

How to create simulator for distributed algorithms written in simple language

I started development of simulator for simulation of distributed algorithms in language C. My work consist of creating simple language for algorithm description and simulator which takes the described ...
7
votes
1answer
329 views

What makes some things easier to parse than others?

I was just reading the Wikipedia page for WebAssembly and it says: “WebAssembly is… designed to be faster to parse than JavaScript” , which got me thinking, what makes a certain language or data ...
1
vote
2answers
163 views

Is there a good single byte delimeter for use with utf-8 strings that isn't a null terminator?

I'm looking for a quick way to split strings containing individual JSON payloads. Currently, I'm using newlines and searching for the newline ASCII character, but I figure if I start using utf-8 this ...
2
votes
2answers
375 views

Best approach parsing input in multiple scenarios

I am developing an API that has one call that accepts a big JSON object. Based on this object, there are 10 possible parsing scenarios, i.e. if field xxxx.xxx is present, go with scenario 5. We ...
5
votes
1answer
377 views

How to parse a simple custom syntax in Go?

I have a limited amount of input types: 34:56 = sensorA#, sensorA#, sensorB# 2:5 = { led# } 66 = otherSensor 2,3,4,5 = greenRelay#, redRelay#, relayA#, relayA# a:b implies range. {name} implies a ...
5
votes
2answers
434 views

SAX-like parser: what is this pattern called?

I wrote a parser for a certain type of binary file with recursive structure. I made its API to be similar to SAX, that is: the parser accepts an object of a specific interface, this interface has ...
3
votes
1answer
596 views

Exception Hierarchy and Use of What Message for Parsing Strings

I have a piece of code parsing a text file line by line. I have to goals: Testing the syntax of the text and extracting information of it. It is quite likely that syntax errors occur so I want to ...
38
votes
4answers
6k views

When to use a Parser Combinator? When to use a Parser Generator?

I've taken a deep dive into the world of parsers recently, wanting to create my own programming language. However, I found out that there exist two somewhat different approaches of writing parsers: ...
5
votes
4answers
1k views

Approach for parsing and indexing very large files

I have been tasked with developing a web based (i.e runs in browser) viewer for a proprietary log file. I have no control over the format of the logs, I just consume them. The log file contains ...