Questions tagged [database]

This tag is for general database questions; if you question is specific to SQL use that tag instead.

Filter by
Sorted by
Tagged with
9
votes
6answers
11k views

Is MongoDB the right choice in my case? [closed]

I'm going to build my first real project in Rails that consist in a web app made of 3 main parts: The static part where no database is used The User registration part which will require a database ...
9
votes
8answers
4k views

Scanning a billion rows in an ultra-fast database

Background A local database contains nearly 1.3 billion unique rows. Each row is indirectly associated with a specific latitude and longitude (location). Each row has a date stamp. Use Case The ...
9
votes
3answers
10k views

How to store large amounts of _structured_ data?

The application will continuously (approximately every second) collect the location of users and store them. This data is structured. In a relational database, it would be stored as: | user | ...
9
votes
3answers
9k views

How should I encapsulate database access?

What are some examples of good class structures used to manage database access? I am a fan of class encapsulation and would prefer the containers (e.g. car) not to perform database tasks. I would ...
9
votes
3answers
272 views

What methods exist for handling DB Change Management? [closed]

I recently began working with version control in my web-development using subversion. This works great for managing the files I develop but does nothing for the changes I sometimes need to make to ...
9
votes
4answers
12k views

What is the best practice for saving TimeZones in the database?

We want to start collecting timezones for each of our addresses in the database. What is the best practice to store the timezones? How would you go about acquiring timezones for the existing address ...
9
votes
2answers
3k views

What is the algorithm for expiring items in key-value storage?

I was thinking on how current key-value storages implement "expire date" for items. Currently I have 2 variants for that in my mind: they don't do anything (keep expired data), and only do check when ...
9
votes
4answers
10k views

Separation of retrieving data and business objects between DAL and BLL Layers

I did some research before posting this question. Among other questions or post, one of those is provided below. I could not get a clear mind how to determine.. Business Objects within a Data Access ...
9
votes
2answers
382 views

When is cloud database right for me?

Some say that DBaaS (database as a service) -- aka cloud database -- is not suitable for Business Intelligence (BI), analytics (OLAP) or archiving. Is this true? More generally, when DBaaS is the ...
9
votes
3answers
977 views

How do you handle constantly changing database dimensions?

For the last two months or so I have been looking for solutions or practices to handle release management within databases. I'm looking for what people view as the best process for handling this. We ...
9
votes
2answers
1k views

To Repository Or Not To Repository

When I first learnt about Domain Driven Design, I was also introduced to the repository and unit of work patterns that once seemed to be top notch for the cool kids that threw SQL queries like ...
8
votes
9answers
391 views

Is it unnecessary to learn the kind of data structures and objects inside sql only because we are using another language to access db indirectly?

Suppose we are using java or python to access the database. Then is it considered a wastage of time and unnecessary to learn the kind of data structures and objects being used inside sql? Please ...
8
votes
5answers
4k views

Why don't databases have good full text indexes

Why don't any of the major RDBMS systems like MySQL, SQL Server, Oracle, etc. have good full text indexing support? I realize that most databases support full text indexes to some degree, but that ...
8
votes
4answers
6k views

Is storing a list of strings in single database field a bad idea? Why?

Recently, I started to work on some legacy system. People that developed it, came to an idea to store list of strings in single field of database table. Lets say that it is an identifier for object ...
8
votes
4answers
843 views

What exactly does it mean that storing “large blobs in the database reduces performance”?

To someone who knows database internals this may be an easy question, but can someone explain in a clear way why storing large blobs (say 400 MB movies) in the database is supposed to decrease ...
8
votes
6answers
7k views

Are HSQLDB unit tests an anti pattern?

HSQLDB is great. It (also) has an embedded mode (no dedicated server needed), which allows for quick prototyping of stuff like Proof of Concepts, and it can also be great in production-ready ...
8
votes
5answers
4k views

How to design a complex REST API considering DB performance?

I've been following some tutorials on how to design REST APIs, but I still have some big questions marks. All these tutorials show resources with relatively simple hierarchies, and I would like to ...
8
votes
6answers
2k views

Am I Unit Testing or Integration Testing my Stored Procedures?

I have had many occasions recently where I have needed to maintain complex stored procedures and functions. These were broken already, usually in fairly subtle ways - there were very few occasions ...
8
votes
5answers
5k views

Is there a downside to using Access as a database?

I have inherited an application that stores data in multiple tables in Microsoft Access; the Access DB is only used for storage, all data processing is handled by the app (VB.net). Is this good ...
8
votes
1answer
461 views

When allowing a user to change passwords, should I generate a new salt?

In my database, I have a column for the hashed password and the salt. My application allows users to change their password. Should I use the same salt for the password, or should I generate a new ...
8
votes
2answers
3k views

Storing a mailing address in a database: What structure should I use for International apps?

I will have international users use my database, but I don't know how the mailing system operates outside of the US. Are the concepts "City", "State, Country and possibly "Zip" sufficient to capture ...
8
votes
3answers
4k views

NoSql databases - Decent tutorial/book that covers the concepts [closed]

Am looking for a good resource on learning the concepts behind NoSql databases. Most things I find are related to a specific technology (MongoDb, CouchDB etc) but I'm after all the concepts behind ...
8
votes
5answers
5k views

Is MySQL viable for small-medium business applications

After having worked with a Microsoft stack - VB.Net, ASP.Net, SQL Server - for the past 10 years, we are now faced with having to develop an application for both Windows and Mac environments. It has ...
8
votes
3answers
881 views

Cascading “Deleted” records that aren't really deleted

Just trying to get some ideas on what people do for this scenario. We have a system database(SQL Server 2008 R2) that has tables and every table has a field we can call "Deleted". It's basically a ...
8
votes
3answers
2k views

Proper Response to HTTP Request When Too Much Data is Requested

I'm building an API for an ad serving platform that will allow you to request tracker data for ad campaigns. Campaigns often exceed hundreds of millions of requests, which means there will be many ...
8
votes
4answers
7k views

What are some best practices for populating and using a test database?

So, I'm developing some database-driven RESTful Java web services, using Hibernate and MySQL. For testing purposes, I'm using the H2 in-memory database. H2 is nice and fast, so this has worked out ...
8
votes
2answers
2k views

Making the switch from MySQL to PostgreSQL?

I've spent the last year becoming really comfortable with MySQL, but due to its increasing trendiness and my desire to homogenize my web-apps with Heroku, I'd like to start using PostgreSQL for my web ...
8
votes
3answers
703 views

Non-transactional Database and Integration Testing

One issue I think I am going to run into with my integration testing is having multiple tests accessing the same database. While this is not a problem now, I know we have multiple applications here ...
8
votes
2answers
3k views

How to safely run database migrations with multiple app instances?

We have an application that has a mix of both fast (< 1 second) and slow database migrations (> 30 seconds). Right now, we're running database migrations as a part of CI, but then our CI tool has ...
8
votes
3answers
2k views

How to manage database schema changes in open source project releases

I manage an open source PHP/MySQL web application used by a few K-12 schools and some colleges. I'm also the project's only developer. While it used to be little more than a source download of an ...
8
votes
2answers
693 views

How to handle complex calculated fields in an ORM

In our API we've got a few central datatypes which need to be "decorated" (so to speak) after retrieval from the database with calculated values. The database is accessed through an ORM which follows ...
8
votes
2answers
436 views

Databases and the CI server

I have a CI server (Hudson) which merrily builds, runs unit tests and deploys to the development environment but I'd now like to get it running the integration tests. The integration tests will hit a ...
8
votes
1answer
2k views

Making API calls with celery

I'm designing a system for a client where the requirements are: they upload a JSON file (one object/line) make a call to an API with the JSON object as the payload record the state (success/failure) ...
7
votes
3answers
460 views

Converting a 1,000,000 rows database

If I have a database that stores the login credentials of 1,000,000 users in PLAINTEXT, how much effort will it take me to md5 hash these passwords?
7
votes
6answers
409 views

Developing cross-DB applications

So far I've always coded using raw/embedded, dialectic SQL. This seems obviously wrong to me because it ties you to concrete technologies so your software can't be deployed to any machine - especially ...
7
votes
8answers
1k views

When did developers start making normalized relational databases?

I work on a pretty old application which was converted from DOS and flat files for storing information to Paradox tables using the BDE then to MySQL relational database. The older devs say that the ...
7
votes
3answers
37k views

How best to store images on a file system [closed]

Developing an application in which users have to upload images. I have not really got much experience with file uploads in an application so I am wondering what the best way is for storing images ...
7
votes
6answers
1k views

Is the “One Description Table to rule them all” approch good?

Long ago, I worked (as a client) with a software which use a centralized table for it's codified element. Here, as far as I remember, how the table look like : Table_Name (PK) Field_Name (PK) Code (...
7
votes
5answers
4k views

How do you off load work from the database?

In at least the web development field, web servers are everywhere but are backed by very few database servers. As web servers get hit with requests they execute large queries on the database, putting ...
7
votes
4answers
608 views

Database design for fields that should be NOT NULL eventually, but may not yet have a value

I have encountered this problem a number of times. Let's say you have a large object that you want to store in the database. Perhaps it's a dating profile for a user, and it has 100 fields. All of ...
7
votes
4answers
3k views

C# database communication technique

A few days ago, I read an answer stating that writing queries within your c# code are long gone. I'm not sure what the specific person meant with the comment but, it got me thinking. At the company ...
7
votes
7answers
7k views

How to develop complex applications

I want to know the approach in developing big complex application regardless of which programming language to use. I want to know how developers make such big applications such as internet banking, ...
7
votes
3answers
3k views

Use of booleans in a database

I'm using Visual Studio 2013, .Net 4.5 and SQL Management Studio 2012. I have a table that tracks offices in my database. On the application side there are a some display rules regarding the layout ...
7
votes
3answers
495 views

Do common relational database systems work 'out of the box'? [closed]

I'm quite new to software development and whenever I come across small projects which involve storing (relational) data I always ask myself if something like a micro-blog (or any other project with ...
7
votes
4answers
846 views

Computational intensive application: loading vs. querying database

My company has a suite of mature (roots go back 15+ years), computationally intensive applications (simulations) which we use to do consulting work. I work on the simulation/consulting side of the ...
7
votes
2answers
6k views

Should I use a plain text database, or SQLite? [closed]

I'm creating a point of sale system for a small shop. I am particularly interested in finding patters from all items purchased, which I will log in their entirety. I'm thinking of even making queries ...
7
votes
4answers
5k views

Are there any real benefits to a DAO layer?

When I first started using Hibernate I heard about all the joy of using a DAO layer. Initially it made sense for my problem: I was going to experiment with Hibernate and later experiment with NoSQL ...
7
votes
4answers
2k views

Best algorithm/practice when creating a search mechanism for your database? [closed]

I have been designing a database where it is very important to provide users with a good search mechanism. So I was wondering what some of the best practices are for using keywords to search over ...
7
votes
5answers
450 views

How best to handle database refactoring within a team?

We are currently usign a roll-forward approach to DB changes, akin to Migrations, where each developer creates and checks in a script that promotes the latest version of the DB to a new state. ...
7
votes
4answers
4k views

Two Database Architecture : Operational and Historical

I thought about an uncommon database structure and wonder if anyone has seen it in use before. It's basically using 2 databases: First database hold only the data that is currently valid Second ...