Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [indexing]

The tag has no usage guidance.

1
vote
0answers
18 views

How to build a HAMT with multi-index keys

By that I mean, forming a Hash Array Mapped Trie with 2 or more indexed fields, such as a User model by email and location name, or email + username + last logged in + isActive. Basically any ...
1
vote
0answers
21 views

Performance impact of JPARepository save() on a large database table with index

We have a few tables with a large amount of data and with indexes on those tables to help in faster retrieval. We are also using Spring Data JPA JpaRepository for adding data to those tables using the ...
2
votes
1answer
31 views

Indexes on a SQL Server fact table

If I have a SQL Server fact table with four dimensions (OrderDate, Customer, Product, Region), my understanding is that it's best to create a non-clustered index per foreign key (dim key column in the ...
0
votes
1answer
121 views

My MySQL database architecture for ecommerce website

I'm making an ecommerce site, using MySQL for storing data. Here is my ERD: CREATE TABLE `admins` ( `id` int(10) UNSIGNED NOT NULL, `shop_id` int(10) UNSIGNED DEFAULT NULL, `name` varchar(75) ...
0
votes
1answer
71 views

Limitations of Tries in Comparison to B-Trees for a Database

I am wondering about how range queries work, and the standard solution is to use B+trees. However, I am a fan of tries as a general data structure and would like to know if they (or variations of them)...
-2
votes
1answer
76 views

Which data structure should I use for implementing a code indexer

I need to write a basic code indexer, which needs to be fast. Should I use an embedded SQLite database for this or should rather rely on a custom data structure, or even flat files as used by ctags? ...
4
votes
3answers
453 views

Is “Array[1]” the first element or second element in the array?

Following the reading of the question Why are zero-based arrays the norm?, I wonder about the terms to use for referring to specific array elements, in the perspective of linguistic reading of ...
1
vote
1answer
2k views

Why is elastic search popular? [closed]

Elastic search is basically about indexing of data. In database world, Multiple indexes can be created on a MongoDB collection Collection in MongoDB can be schema-less. In MongoDB, BSON encoding ...
0
votes
3answers
196 views

Strategy to update search index after fixing index generation

Describing the situation I'm working on an application (based on the Spring Framework) using a search index (lucene if that matters) to make content of that application searchable. Documents are ...
1
vote
1answer
300 views

What is an indexer?

The C# docs have a page on indexers, which appears to use "indexer" to refer to the construct required to enable instances of a class to be accessed via square bracket notation. Indexers allow ...
0
votes
0answers
256 views

Managing an index of file metadata on a network share

What is a good way of setting up a "shared index" of file metadata, when there can be no shared process such as a database server? I'll explain the scenario: A server contains M (say 10000) large ...
3
votes
0answers
115 views

What is a good way to structure a timespan-oriented index?

I'm finishing off work on a complex rule-engine for Hotel Rates with real-time queries. There is a lot of conditions addressing intersections of periods for options, restrictions and policies. All of ...
5
votes
4answers
800 views

Can someone explain the technicalities of MapReduce in layman's terms?

When people talk about MapReduce you think about Google and Hadoop. But what is MapReduce itself? How does it work? I came across this blog post that tries to explain just MapReduce without Hadoop, ...
5
votes
4answers
2k views

Approach for parsing and indexing very large files

I have been tasked with developing a web based (i.e runs in browser) viewer for a proprietary log file. I have no control over the format of the logs, I just consume them. The log file contains ...
1
vote
2answers
137 views

Use (numeric) IDs over names as unique key?

I have a set of data (assume they are objects) with unique immutable names, like this: class Datum { final string name // other fields } Considering that: I don't need to support rename. (...
2
votes
1answer
2k views

Is indexing foreign keys a good practice?

Looking at DB tables created by a different developer I have noticed that whenever a table had a forein_key_id field/column, it was always an INDEX/KEY. I am not sure if it was manually created, or ...
2
votes
1answer
100 views

Indexing Algorithm That Avoids Duplicate Lookups On An Uncontrollable Dataset

How to index a massive, randomly selected, uncontrollable, constantantly changing dataset? Imagine you want to index all of the snow particles in a giant snowglobe that is constantly being shaken. ...
1
vote
2answers
371 views

Creating a web application with full text search on dynamic data

Even after thorough requirements engineering we end up with users wanting to attach 'notes' to their otherwise well-structured data records, in other words: arbitrary key-value pairs. Their primary ...
7
votes
4answers
1k views

What is the origin of counting from zero in programming languages?

This is a question which I have wondered (and been asked) about for a long time. In (most? all?) programming languages, an index begins at zero for an array, string, etc. I recognize it became ...
0
votes
1answer
104 views

Best Azure Solution for Complex Search Index

I need to perform quick searches against a combination of tags while including date ranges: Example: Users who have requested notifications who did not respond to a notification sent at least 3 ...
4
votes
1answer
159 views

Efficient Value-Lookup in List of k-Tuples

I've encountered a problem in a personal project that I think could be solved by a particular data structure but I'm not sure what. The problem is as follows: Given a set of k-tuples, provide an ...
-1
votes
2answers
107 views

Does sql use hastables for indexes? [closed]

Are there other ways of indexing?, Which are the more used? Does sql have an standard for indexes, it uses hash tables?
0
votes
3answers
514 views

Bits - Least-Significant/Lowest is 0th or 1st; zero or one indexed

Question Is there a rough consensus if the bitmask 0x01 is properly said to have the "zeroth" bit set, or the "first" bit set? If there isn't rough consensus that there's a generally right answer, ...
-1
votes
1answer
78 views

Is there a generally accepted definition of “Secondary Index” independent of DB product? [closed]

Is the definition for "Secondary Index" anything more specific than just "Any index that is not the primary index"? EDIT: Here is some research I have done: Search Google. I evaluated the first 20 ...
1
vote
1answer
123 views

Search by Location and Keywords [closed]

I have data set with ~5M entries/rows (~2GB). Every entry contains a location field (lat/lon coordinate-pair) and n keyword fields (keyword-1, keyword-2, ... keyword-n). The keyword-fields can all ...
1
vote
0answers
206 views

Saving and retrieving multiple cached json/txt data files

I am working on some numerical programming and need to generate the results for a model given a variety of input parameters. Since the model takes a while to run, I was planning to save the data to ...
25
votes
8answers
4k views

Should my sequential collection start at index 0 or index 1?

I am creating an object model for a device that has multiple channels. The nouns used between the client and I are Channel and ChannelSet. ("Set" isn't semantically accurate, because it's ordered ...
-1
votes
1answer
125 views

node deep file indexer module cannot go deep [closed]

I'm trying to make a file indexer by node.js. The program is supposed to index files and folders in an array and also check inside folders and add all sub folders and files I wrote: fs.readdir(...
3
votes
2answers
174 views

Efficient range search for pair of numbers

Assume we have large list of pairs: struct {x: double, y: double} pair; vector<pair> What is the most effective way to find all pairs where (x1 < x < x2) AND (y1 < y < y2)? O(n) ...
2
votes
1answer
814 views

Finding the closest n points to any arbitrary point in two dimensions (r-tree, quadtree, spatial index)

I have a distribution of two dimensional point objects. How is it possible to find the nearest N number of points to any given point without iterating over the entire collection of points (and only ...
12
votes
5answers
2k views

How come the computer doesn't have to read the entire table when the column is indexed? [duplicate]

Let's say a table with two columns has 100 quadrillion records. And I want to find a record that has column #2 equal something. If column #2 is indexed it returns the result immediately, but if it's ...
1
vote
1answer
76 views

Difficulty in deciding correct data structure

Two objects are interacting (Object Alpha, Object Beta) Each contain a point (x,y) which will be used to make comparisons, among other things. Object Alpha's point (x,y) attribute is dynamic and ...
61
votes
10answers
6k views

Is it premature optimization to add database indices?

A colleague of mine today suggested that we go through all of the queries in our application and to add indices accordingly. I feel this is premature optimisation because our application is not even ...
0
votes
1answer
711 views

Full Text Indexing Strategy for MS Excel Documents

Background As part of a broader application that allows users to search thousands of MS Office documents on a private network, I need to index and make searchable Microsoft Excel files. My basic ...
10
votes
5answers
1k views

Quadtree with duplicates

I'm implementing a quadtree. For those who don't know this data structure, I am including the following small description: A Quadtree is a data structure and is in the Euclidean plane what an ...
2
votes
4answers
3k views

Algorithm for indexing strings in “list”

Imagine I have file called strings.dat. Inside this file there are a lot of strings, for example: one million. Strings are sorted. Now I want to find a specified string, so I can write method like ...
1
vote
2answers
947 views

Data retrieval and indexing

I have around 800,000 rows of data stored in the boost shared memory from the database. The data are in the form: Id Color Length Size 1 1 2 ...
4
votes
3answers
3k views

storing and retrieving millions of documents using c#

I am working on an integration project, where my “app/web service” will sit in the middle serving documents. Basically, a request is sent with the document id as part of the query string, I check if ...
3
votes
3answers
887 views

Real world analogy for a clustered index

A db index is analogous to a table of contents. This helps me understand db index in an easy way. My question is are tehre any real world analogies for a clustered index?
0
votes
1answer
3k views

What exactly is a Search Engine Indexer. Where to start building one? [closed]

For one of the projects at uni we were given the task to create a custom, niche search engine. My colleges and I split the tasks amongst each other so that we can tackle the overall project more ...
14
votes
6answers
3k views

Do You Still Need Indexing After Database Normalization

After you have done a good normalization, do you still need to index the table? How is this going to affect the performance? Will it even affect the performance in some way after a good normalization? ...
10
votes
1answer
422 views

lightweight document indexing to handle less than 250k potential records

Recently I've found myself chafing at the limitations of document indexing engines. I was developing a small website that needed some fairly robust searching capabilities but due to their hardware ...
-2
votes
2answers
242 views

Using database Indexes [closed]

This might sound like a naive question but when are database table indexes in mysql required? How do such indexes affect performance and why?
5
votes
2answers
973 views

Fast set indexing data structure for superset retrieval

I am given a set of sets: {{a,b}, {a,b,c}, {a,c}, {a,c,f}} I would like to have a data structure to index those sets such that the following "lookup" is executed fast: find all supersets of a given ...
0
votes
1answer
981 views

Wildcard user in select statement on indexed varcharcolumn

This question is based on some testing I've done recently in order to better understand SQL indexing system. Table with 500k entries, InnoDB engine This is a simple select query on a varchar field. ...
2
votes
6answers
10k views

How many indexes is too many indexes?

I have a table with about 37 indexes. Would that be considered too many? What are some best practices regarding indexing? EDIT: It is warehousing data, updated daily with new transactions. There are ...
8
votes
1answer
726 views

How do I efficiently store all OpenStreetMap data in an indexed way?

I have a PBF file that contains the following information about a country: Nodes, each with their own longitude, latitude and properties; used to store points in a 2D space. Ways, each with their ...
0
votes
2answers
199 views

Dissection of Google indexing services

There are more than a few questions that hop into mind when someone thinks about Google's indexing services. Jeff Atwood wrote about them at The Elephant in the Room: Google Monoculture and Trouble In ...
29
votes
7answers
98k views

How to quickly search through a very large list of strings / records on a database

I have the following problem: I have a database containing more than 2 million records. Each record has a string field X and I want to display a list of records for which field X contains a certain ...
5
votes
1answer
9k views

Database design for tagging system

I'm creating a system where items will have one or more tags. Now there will be an item table that will store each item, and a tags table, that will store all tags used in the system. My question is:...