Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [parsing]

Analyzing (un)structured data to convert it into a structured, normalized format.

0
votes
2answers
91 views

Should a SVG be converted to JSON to be stored in the database?

Consider an application where you can create and edit a SVG file (e.g. draw.io). How should the SVG be stored in the database? With SVG (image/svg+xml) Thus no further processing is required before ...
0
votes
2answers
69 views

Understanding Context Free Grammar using a simple C code

I'm trying to understand the difference between terminal and non-terminal values in a real language. I wasn't able to find enough examples on real language CFGs on the internet, most examples are ...
1
vote
3answers
85 views

Catch statements with other Business Logic in Parsing Files

I am writing a simple program to parse a text file and place into Generic List. Sample Text: 1,Joe,CA,58,2 2,Matt,TX,63,5 Sometimes, there may be an error, with missing data in the file 1,Joe,CA,...
0
votes
2answers
106 views

Is there a name for these diagrams that show the flow of a valid value [duplicate]

Here's an example taken from http://json.org/ Is there a name for this kind of diagram?
0
votes
0answers
13 views

Design of a schema parser in order to validate files according to that schema?

I am trying to parse files written in the EXPRESS data modeling language. These files are used to define data structures of files used for the exchange of product manufacturing data. In my case, this ...
0
votes
0answers
12 views

General structures / algorithms for parsing / interpreting a semantic query efficiently

If you have a semantic/meaning graph data structure built up, and you get a new word as part of a sentence, the question is, how it optimally determines the meaning of that new word. It seems that, ...
1
vote
0answers
65 views

What obstacles should I expect in building universal Abstract Syntax Tree's manipulation tool?

I am trying to build code generation and transformation tool and hence I need software for Abstract Syntax Tree manipulation for at least three languages (Coq, Haskell, Java). So far I have managed to ...
0
votes
0answers
58 views

Proper Architectural Structure to Parse files and import to database

After reading this, what is the proper architectural structure to parse files and write into a database? https://www.codeproject.com/Tips/806934/How-to-Return-the-Contents-of-a-CSV-file-as-a-Gene How ...
1
vote
1answer
118 views

Parsing huge XML files

our startup is working on an integration for various customers. All of the customers are required to provide their data in one unified way = we chose XML files because of legacy compatibility and ease ...
-2
votes
3answers
98 views

Parsing Complex Context-Sensitive Text

I am trying to learn how to handle complex context-sensitivity in parsing. Say you have this simple example of code: var x = 10 var y = x + 10 var z = y + 10 var a = z + 10 var b = a + 10 var c = b + ...
0
votes
1answer
82 views

What algorithm to use to parse expressions in this simple language?

I don't want to include the entirety of my language, but I think the relevant part is: <expr> ::= <logical> | <comparison> | <constant> | <...
0
votes
0answers
37 views

Is re-using parts of older mapper/parser versions advisable?

Context I'm working on an app that uses the JSON format for persistence on a web server. The app is in production and features are being added over time, so from time to time there's some additional ...
0
votes
1answer
221 views

Do any programming languages let you use other languages without restriction within them?

This may be a stupid question, and it would certainly take one Hell of a lexer, but do any extant programming languages allow you to do something like: c# (1.2) { // c# code } Perl (5) { # ...
2
votes
1answer
148 views

One data structure to both parse and stringify

You can create a data structure such as a Parsing Expression Grammar (PEG) that will be used to parse: string -> object You can then write a function that iterates through the object's properties ...
1
vote
1answer
75 views

Backend - own database vs live parsing

I am developing a mobile app that depends on some nearly static data. I will gather that data initially from an existing website by doing some HTML parsing. I could now create a database from that ...
0
votes
1answer
121 views

AST design: Call is both expression and statement?

I'm designing the AST for a compiler and found that I don't actually know how to represent the Call node. Currently, the relevant part of the AST looks like this (where the arrows denote inheritance):...
1
vote
4answers
279 views

Are parsers a special case in unit testing?

Context While trying to get my parser classes under test, I noticed a common challenge for (unit-)testing them: They have only one public method, with a string as input and the parsed class as output....
3
votes
0answers
34 views

Parse track meet output file result

I am building out a parser for a well-known track & field meet management software output file (sample output). I can parse this manually, and put items into my object model (overview below), but ...
5
votes
5answers
640 views

How to Unit-Test a parser of a file?

I'm implementing a metadata parser of image files from all formats. I want to write tests for it. One trivial way to do so is to have test image files of all formats as a resources for the tests, and ...
2
votes
2answers
110 views

Designing for the validation of phone numbers

Let's say we have a registration system that finalises the creation of an account by SMSing the user their new account details. This system could be used by people all around the world, so let's say ...
1
vote
1answer
136 views

Can I get information about an SQL SELECT query (such as the tables it accesses) using C#?

The c# application I'm working with executes many T-SQL SELECT queries that are stored as strings in a database. These queries almost all involve selecting data from and joining with several tables ...
0
votes
1answer
280 views

Grammar by example

Is there an automated approach to generate a grammar (which could be used later in a compiler tool such as ANTLR or similar) from given examples of a language? With more detail: assumed a technical ...
3
votes
1answer
81 views

What approaches can I take to figure out the “relevancy” of certain terms in a string?

I'm not even sure "relevancy" is the most accurate word, so I'll just describe the problem: I'm building an app that needs to somehow parse product descriptions from a popular website (let's just say ...
3
votes
1answer
98 views

Meaningful response to the user after his uploaded CSV was processed?

A user uploads a CSV to the server to be inserted into the database and I would like to return a meaningful response to this user. My thoughts : { inserted : x records discarded : y records } ...
0
votes
1answer
93 views

Optimal pattern for parsing delimited (e.g. new line) network stream

I am looking for optimal for maintenance/easy of programming and elegant patterns to parse delimited network streams, e.g. protocols like SMTP. Some of the concerns I've always had are: how to react ...
3
votes
1answer
187 views

Is it worth parsing an infix algebraic expression to postfix and then an expression tree?

I am trying to make a simple expression parser, in which users type an expression in infix notation (for example ln((1+2)(3-4))) to apply some calculations. To make this possible I need to tokenize ...
-3
votes
1answer
113 views

Multiple variable declaration, multiple variable assignment, context-sensitive 'in' statement

Lately I've been playing with writing my own programming language, following the excellent Crafting Interpreters book but I've hit something of a snag. I'd like to extend the parser to accept ...
-1
votes
1answer
128 views

How do different file types generally store data?

I am working on a project, for which I want to create my own file format to store large amounts of data. I am trying to decide how that file format should be built to be as efficient as possible. The ...
3
votes
1answer
325 views

Which is a more efficient approach to decoding escape sequences in text?

I'm working on parsers that not only process delimited content, but also escape sequences within certain portions of that content. I'm contemplating the efficiency of several approaches to ...
0
votes
1answer
148 views

Designing syntax tree derived node classes

Let's say I have an abstract Node class and BinaryOperation class derived from Node and three classes IntegerConstant, RealConstant and CharacterConstant also derived from Node. The general design ...
1
vote
1answer
272 views

Desired pattern for expression parsing into RPN

Im writing a interpreter/compiler I have encountered a problem which I have solved before, but not optimally perhaps. The problem goes like this, expressions can express alot of things like scopes, ...
2
votes
2answers
188 views

Is implementing operator overloading with variable arity possible?

Implementing operator overloading for e.g. + (plus) isn't terribly difficult if you know it's a binary operator. One can just parse expression + expression But what if the programmer can choose ...
0
votes
3answers
1k views

What approaches can be used to convert existing JSON structure to existing XML structure?

I am working on a component for an enterprise message queue that consumes a message from a queue and sends it as a text message. The message is picked up by my consumer in JSON format (I do not have ...
1
vote
4answers
351 views

Is it common to write File Parsers in higher languages like Java?

I work with an older framework (15 years+) at work which is fully built on Perl. Its main tasks are calling external programs via command-line calls, parsing their output and presenting some of the ...
4
votes
2answers
313 views

Correct terminology for if-blocks

I am using lex/yacc to parse a simple language which allows 'if blocks': IF something something [ELSE IF something][ELSE] END IF and also other nested logic under a 'DEFINE' statement: DEFINE ...
0
votes
0answers
50 views

Parsing nested structure from facebook pfff - how to decide on parser

I'm playing around since a while with building a little open source project based on the output of facebooks pfff tool, that can parse multiple programming languages and output some kind of unified ...
5
votes
3answers
250 views

Proper separation between lexing and parsing

I am currently writing a parser which, given a source file, turns it into an AST of some language, respecting the idiomatic process of lexing and then parsing using well-known parser generators (think ...
1
vote
0answers
80 views

Concurrently parsing records in a binary file in Go

I have a binary file that I want to parse. The file is broken up into records that are 1024 bytes each. The high level steps needed are: Read 1024 bytes at a time from the file. Parse each 1024-byte "...
4
votes
3answers
244 views

How to create simulator for distributed algorithms written in simple language

I started development of simulator for simulation of distributed algorithms in language C. My work consist of creating simple language for algorithm description and simulator which takes the described ...
7
votes
1answer
328 views

What makes some things easier to parse than others?

I was just reading the Wikipedia page for WebAssembly and it says: “WebAssembly is… designed to be faster to parse than JavaScript” , which got me thinking, what makes a certain language or data ...
1
vote
2answers
163 views

Is there a good single byte delimeter for use with utf-8 strings that isn't a null terminator?

I'm looking for a quick way to split strings containing individual JSON payloads. Currently, I'm using newlines and searching for the newline ASCII character, but I figure if I start using utf-8 this ...
2
votes
2answers
351 views

Best approach parsing input in multiple scenarios

I am developing an API that has one call that accepts a big JSON object. Based on this object, there are 10 possible parsing scenarios, i.e. if field xxxx.xxx is present, go with scenario 5. We ...
5
votes
1answer
356 views

How to parse a simple custom syntax in Go?

I have a limited amount of input types: 34:56 = sensorA#, sensorA#, sensorB# 2:5 = { led# } 66 = otherSensor 2,3,4,5 = greenRelay#, redRelay#, relayA#, relayA# a:b implies range. {name} implies a ...
5
votes
2answers
411 views

SAX-like parser: what is this pattern called?

I wrote a parser for a certain type of binary file with recursive structure. I made its API to be similar to SAX, that is: the parser accepts an object of a specific interface, this interface has ...
3
votes
1answer
531 views

Exception Hierarchy and Use of What Message for Parsing Strings

I have a piece of code parsing a text file line by line. I have to goals: Testing the syntax of the text and extracting information of it. It is quite likely that syntax errors occur so I want to ...
37
votes
4answers
5k views

When to use a Parser Combinator? When to use a Parser Generator?

I've taken a deep dive into the world of parsers recently, wanting to create my own programming language. However, I found out that there exist two somewhat different approaches of writing parsers: ...
5
votes
4answers
1k views

Approach for parsing and indexing very large files

I have been tasked with developing a web based (i.e runs in browser) viewer for a proprietary log file. I have no control over the format of the logs, I just consume them. The log file contains ...
0
votes
0answers
54 views

Is there a name for this grammar transform (or a more general form of it)?

I'm working on removing left recursion from a grammar, and of course started with the algorithm described on Wikipedia. As advertised, it unfortunately exploded into a much larger grammar that was ...
5
votes
2answers
826 views

Postfix vs Prefix

I have read at a few places, that Postfix is easier to evaluate & easier to convert to Code than Prefix. I understand that if you parse a prefix expression from left to right, you might not be ...
0
votes
1answer
151 views

Is MVC (or variations) the right pattern when retrieving data, then parsing and displaying it?

Due to these answers, I've decided to implement MVC (or a variation of it) into my application. But the more I read the more confused about this topic I get. Some people say the business logic belongs ...