Questions tagged [natural-language-processing]
Natural language processing draws knowledge from a diverse collection of fields including computer science, linguistics, and statistics in order to extract pertinent information from the spoken or written word.
43 questions
2
votes
2
answers
5k
views
Separating words in a string
How do I separate words in a string?
In the following I have a random sample of words in a string extracted from text file with over a million words.
Here's the string:
"intervene Pockets ...
-2
votes
3
answers
181
views
Its possible to distinguish between functional and non functional software requirements by computer software? [closed]
I am trying to do a computer program that can distinguish the functional and non-functional requirements written in Arabic languages from the Linguistic structure, so I am I look for the things that ...
- 19
-2
votes
1
answer
149
views
Is there a way to encode binary into natural language?
I was wondering if there was a way to encode binary data, such as Crypto addresses into a natural language format, in a similar fashion to how what3words.com can encode locations into three easy-to-...
- 16.3k
-4
votes
1
answer
46
views
How to make comprehension questions from text?
Lets say I wanted to autogenerate questions for a wikipedia page as a sort of comprehension exercise, how would i best go about that.
Using machine learning would be necessary but any other things i ...
- 530
-1
votes
1
answer
52
views
Equal transformations on both indexed content and query content before a search is attempted
In search engine indexing, a body of text is often processed before it is indexed. A common example is stemming, were words are reduced to their root form (plurals are dropped, tense is normalized). ...
- 80.6k
3
votes
1
answer
190
views
Designing the schema for a database of Spanish language words?
For a project I'm working on that will help people learn Spanish, I would like to create a standalone service to handle the retrieval of data about words. For this, I've captured and codified data ...
CommunityBot
- 1
4
votes
2
answers
608
views
Database structure for word co-occurrence frequencies in a large corpus
I would like to store the frequencies with which words co-occur with each other over a variety of contexts in a large (> 1 billion tokens) text corpus. I need to store the word pair, the type of co-...
3
votes
0
answers
94
views
Software design strategy for a machine learning tool that outputs a subset of the text input (Information Extraction)?
Let's say I have thousands of pdfs that are each about 30k words written in conversational English. In each of the pdfs there is a name / names of a person/people who snowboard. There are also many ...
- 39
3
votes
1
answer
86
views
What approaches can I take to figure out the "relevancy" of certain terms in a string?
I'm not even sure "relevancy" is the most accurate word, so I'll just describe the problem:
I'm building an app that needs to somehow parse product descriptions from a popular website (let's just say ...
- 79.8k
0
votes
2
answers
86
views
Divergent Bodies of Truth, Relationships of Concepts
A recent endeavor of mine requires asking users how related two concepts are. Example: Kiwi and Fruit. Identical? Fairly Similar? Rather different? Unrelated?
It occurs to me that for certain ...
- 80.6k
0
votes
1
answer
518
views
What is the best way to summarize a sentiment value for a paragraph of text based on the sentiment value for the sentences within it?
So I am using Stanford CoreNLP in my project. I have data which consists of reviews of products on a forum. I need to be able to assign a sentiment value to a given review. CoreNLP allows you to ...
CommunityBot
- 1
1
vote
2
answers
330
views
Automatic summarization - whole text or paragraphs
I am writing a tool that will give users the ability to summarize text content on a webpage, by highlighting the text that they wish to get summarized.
So far, I've received results that I can work ...
3
votes
2
answers
670
views
Available options for classifying words in text?
I am researching ways to classify words in text and I'm wondering what options there are and which are best suited to this job. I'm mostly interested in keywords which are most often nouns.
So far I ...
CommunityBot
- 1
4
votes
1
answer
500
views
Sentence Tree vs. Words List
I was recently tasked with building a Name Entity Recognizer as part of a project. The objective was to parse a given sentence and come up with all the possible combinations of the entities.
One ...
- 84.2k
3
votes
2
answers
675
views
How to build a class for comparing words in a lexical dictionary?
I'm wanting to create a class that stores words in a set so that I can see if a word belongs to that set or not. I'm not wanting to build ever set every time I instantiate the class, so I'm using what ...
CommunityBot
- 1
3
votes
2
answers
300
views
Is polymorphism appropriate for modeling natural language form (structure), or is there something better?
Let's take French and Japanese adjectives as simple examples.
French adjectives have gender (masculine or feminine) and number (singular or plural), whereas Japanese nouns have neither. However, ...
- 80.6k
0
votes
1
answer
190
views
Which strategy to use in Client-Server messaging to trigger specific behavior in them?
I'm developing a virtual assistant to manage transactions in a business. I'm using an external service to handle Natural Language and
get intents, actions and parameters from my user requests.
The ...
- 80.6k
3
votes
1
answer
249
views
Tasks incorrectly categorized with online text classifiers
Context: Finding company to do the job
The use case of an application I want to write is that a user will describe a task to be performed by some company, e.g.,
I want to move 10 boxes (30x40x50cm)...
CommunityBot
- 1
17
votes
2
answers
706
views
How to find hard to misspell given names?
Here is a question that I believe could be solved with some data mining and a sophisticated algorithm, but I don't quite know how. Any pointers as to what data sources to use and what algorithm to ...
- 80.6k
2
votes
2
answers
2k
views
How to create an efficient implementation of a dictionary of synonyms?
How do I create a dictionary of synonyms that is efficient in terms of retrieving synonyms of a word?
Basically, I want to use some sorts of data storage to store groups of words, and enable an end-...
- 80.6k
11
votes
6
answers
3k
views
How to teach a script to detect sarcasm? [closed]
I'm currently building a fun script, that basically matches given phrases and gives a predefined response based on the match-points. You can ask it to retrieve some information based on live feeds, ...
- 20.8k
1
vote
1
answer
86
views
Extract the main entity from a body of text for text categorisation?
I'm trying to categorise products based on various fields of data. I've had some success just matching search terms in the product names, but this naive approach doesn't work when it comes to larger ...
CommunityBot
- 1
1
vote
1
answer
739
views
What type of "spam filter" algorithm will tokenize characters for non-exact matches?
I was having a look at a basic example of spam filtering using a logistic regression algorithm and an answer posted on this Stack Overflow question: https://stackoverflow.com/questions/29291263/mllib-...
CommunityBot
- 1
0
votes
1
answer
1k
views
What is the purpose of chunking strings in natural language processing?
I see many examples of libraries that use a 'chunker' and also people asking how to write chunkers, but what is this for and why do we need it? Isn't it enough to split a text by whitespace or other ...
- 39.2k
6
votes
1
answer
264
views
Identifying plagiarized jokes? [closed]
I'd like to be able to identify duplicate jokes posted on a website. I can build up a reasonably large database of previously-posted jokes, and then I'd like to look at each new joke as it comes in ...
CommunityBot
- 1
8
votes
2
answers
1k
views
How do personal assistants typically generate sentences?
This is sort of a follow up to this question about NLG research directions in the linguistics field.
How do personal assistant tools such as Siri, Google Now, or Cortana perform Natural Language ...
CommunityBot
- 1
-2
votes
1
answer
373
views
Using NLP can we create a summary of functional specification [closed]
Just thinking that is it somehow possible by using Language processing APIs to extract a must have points or an intelligent summary from documents like functional specifications ..
- 9,800
1
vote
4
answers
1k
views
How to automatically connect letters into words? [closed]
I would like to connect letters into meaningful words? E.g.
input:
T E ST IN G WE B A P P LIC AT IO N S
output:
TESTING WEB APPLICATIONS
what kind of task is this? A task in NLP, or just some ...
- 3,021
3
votes
3
answers
1k
views
NLP - Queries using semantic wildcards in full text searching, maybe with Lucene?
Let's say I have a big corpus (for example in english or an arbitrary language), and I want to perform some semantic search on it. For example I have the query:
"Be careful: [art] armada of [sg] is ...
- 17.8k
3
votes
4
answers
4k
views
Programming a chatterbot - understanding language [closed]
I hope this is an answerable question. Let me give it some context:
I am a psychology student and a programmer. I'm going to look into creating a artificial intelligence in the form of a chatterbot. ...
- 20.8k
144
votes
14
answers
15k
views
Simple method for reliably detecting code in text?
GMail has this feature where it will warn you if you try to send an email that it thinks might have an attachment.
Because GMail detected the string see the attached in the email, but no actual ...
CommunityBot
- 1
0
votes
1
answer
200
views
What is latent parsing in NLP?
I'm reading a paper describing a NLP work, and I hardly catch the concept of a term, "latent parsing".
Original paper: http://web.stanford.edu/~angeli/papers/2013-acl-temporal.pdf
The figure in the ...
4
votes
1
answer
161
views
How can I test a search engine for an uncommon human language?
We are writing a search engine from scratch in a quite uncommon language, Aramaic, mostly for learning purposes but also because few resources are available in given language.
The engine is/will be ...
CommunityBot
- 1
0
votes
2
answers
222
views
How could an expert system get some meaning from a sentence? [closed]
How could an expert system tell following scenarios apart? How could we include some meaning of a sentence, if at all?
A machine needs 1 day to produce x units, and,
we know that two machines like ...
- 3,021
5
votes
6
answers
6k
views
Are there any algorithms for splitting or combining words into their more common form?
Are there any existing algorithms which can look through a list of words and split or combine words into their more common form?
For example, I have a list of many business names in the health care ...
- 21
0
votes
1
answer
437
views
Natural Language to Search Criteria - Date Ranges
Consider an application that stores a set of records that contain:
Description
Cost
Purchase Date
I'd like to be able to allow users to utilize natural language to search the dataset.
For example, ...
- 18.2k
1
vote
1
answer
216
views
How to do simple natural language categorization [closed]
How would you go about parsing a sentence like
"Bought two kilos of steak from Acme supermarket" into a data structure like the following JSON representation:
{
item: {name: "steak", tags: "meat,...
- 4,646
2
votes
1
answer
354
views
How can I distinguish between one-worded expressions and two-worded expressions?
I was told to create two lists concerning most frequently used words from a plain text (10 Mb arbitrary texts) as monograms (for single worded expressions such as human, water, is) and bigrams (for ...
- 11
12
votes
2
answers
2k
views
Persisting natural language processing parsed data
I've recently started experimenting with natural language processing (NLP) using Stanford's CoreNLP, and I'm wondering what are some of the standard ways to store NLP parsed data for something like a ...
- 1,508
10
votes
3
answers
8k
views
What algorithm(s) can be used to achieve reasonably good next word prediction?
What is a good way of implementing "next-word prediction"? For example, the user types "I am" and the system suggests "a" and "not" (or possibly others) as the next word. I am aware of a method that ...
- 20.8k
1
vote
2
answers
3k
views
How to create a Semantic Network like wordnet based on Wikipedia?
I am an undergraduate student and I have to create a Semantic Network based on Wikipedia. This Semantic Network would be similar to Wordnet(except for it is based on Wikipedia and is concerned with "...
CommunityBot
- 1
1
vote
1
answer
3k
views
Is Agile Development used in Machine Learning and Natural Language Processing?
I've been developing web apps for a while now and it is standard practice in our team to use agile development techniques and principles to implement the software.
Recently, I've also become involved ...
- 631
7
votes
2
answers
211
views
Guess if a time is AM or PM
I'm currently in the process of writing a human date parser. By human date, I mean it should be able to interpret strings as "tomorrow at 2" and return a valid date depending on the current time.
The ...
- 231