Questions tagged [data-warehouse]
The data-warehouse tag has no usage guidance.
33
questions
-2
votes
1
answer
89
views
What is a route I could take when considering setting up a data warehouse for a small business? [closed]
I am working for a medium-sized development and construction company and we are looking to upgrade our data and BI. Currently, all project data is managed via Excel (data, project management, and ...
0
votes
3
answers
226
views
In data engineering, why is data integrity checked on the DW rather than on the data sources?
I'm a software developer and new to data engineering, so this may be a newbie question, but I'm wondering why data integrity checks (for instance, dbt tests) are ran on the data warehouse, rather than ...
- 243
0
votes
0
answers
43
views
Using cloud data warehouse as back-end for website
I am building a website, where the main data source is Snowflake, a cloud data warehouse.
This cannot be changed, but we can add other technologies.
There is a lot of data in Snowflake, and the ...
- 101
-2
votes
2
answers
415
views
What happens after the ETL process?
I have thousands of .csv files with the same structure and, in most of the cases, some column values are the same ones recurring. Each file represents a report on some structures, with numeric ...
0
votes
1
answer
95
views
Attributes at the fact table in dimensional data model
I know that a fact table in dimensional data model consists of measures and flags for an event or transaction. I also understand that it may contain date of the event. I have to design on fact table ...
0
votes
1
answer
378
views
How to keep a data warehouse updated?
Suppose there is a system ( like an ERP ) that writes to a database ( not too big, less than 100GB ). You need to export the data from this database to a data warehouse ( like RedShift or BigQuery ) ...
- 183
0
votes
1
answer
280
views
Do Data Warehouse standards allow foreign key constraints at a dimensional model?
Is it true that we never enable foreign key constraints in the dimensional model of a data warehouse? If yes, then what is the rationale behind that?
As per my research:
Some experts told me in a ...
- 33
-3
votes
1
answer
88
views
what is event based data integration? [closed]
Please help me to understand what is event based data integration in simple layman term with some examples?
How it is different from other form of data integration.
Some sample use cases will be ...
4
votes
2
answers
1k
views
How should a data warehouse be maintained for a quickly changing schema
I am currently in a process of maintaining a data warehouse for a quickly growing start up company. There is a lot of reporting demands from the clients, and this is usually handled by a data ...
- 151
1
vote
1
answer
797
views
Load for Date dimension table of a warehouse
I have a general question about loading data into a data warehouse (DW).
This is basically a followup to an older question of mine.
I have a general understanding problem about fill the [Date] ...
- 159
3
votes
2
answers
156
views
Enterprise application warehousing and relational database
I have a general question about design pattern for an enterprise application.
I read a lot about it but its actually hard to find an answer because most you find it rater about how to design a data ...
- 159
1
vote
3
answers
153
views
How to make a data warehouse of a single full Database?
After learning about the data warehouse concepts whole day, I couldn't found a source that can actually say when you should not do data warehousing. In most tutorials I read, it says that DW is for ...
- 119
0
votes
1
answer
99
views
Lightweight data mining + organization & visualization
I'm looking to do some simple data mining that consists of going once per day to a single page and collect the following information:
List of movie theaters
Movies today on each theater
Session times ...
- 11
2
votes
1
answer
843
views
Indexes on a SQL Server fact table
If I have a SQL Server fact table with four dimensions (OrderDate, Customer, Product, Region), my understanding is that it's best to create a non-clustered index per foreign key (dim key column in the ...
- 23
26
votes
8
answers
2k
views
Is there a programming paradigm that promotes making dependencies extremely obvious to other programmers?
I work in a Data Warehouse that sources multiple systems via many streams and layers with maze-like dependencies linking various artifacts. Pretty much every day I run into situations like this: I run ...
- 369
1
vote
3
answers
605
views
Categorizing columns in a database table
I wanted to know how do people generically think of, in a data base table:
Columns that people are expected to use mostly for grouping expectable of groupability.
Columns that people are expected to ...
- 6
6
votes
4
answers
2k
views
How and when to choose between Data Consolidation/Data Integration and Data Synchronisation?
I am having some confusion as to how to decide what to choose between Data Consolidation/Data Integration and Data Synchronisation? I know that:
Data Consolidation/Data Integration: refers to the ...
- 161
6
votes
3
answers
2k
views
Designing a data warehouse: action or status based
I am designing a data warehouse for a sales platform. There exists a group of users that offers and a group of users that demand entities.
The production data base holds two sources of information:
...
- 227
2
votes
0
answers
60
views
How are values such as monthly cost stored in a data warehouse
Would a value such as monthly cost from source data be represented as a fact?
If yes, how would a date dimension table be designed around that fact?
- 21
4
votes
2
answers
2k
views
Am I using Data Warehousing wrong?
I work for a company that has several products, and several databases and tons of data.
To sorta merge all this data what we've been doing moving some of our data over to Amazon Redshift (a data ...
- 363
1
vote
3
answers
377
views
Are Data Warehouses bespoke products that are tailored towards the specific needs of one single organisation?
Are Data Warehouses bespoke projects that are developed to meet the needs of a specific organisation or can you buy off the shelf products that would meet the needs of many organisations?
I am ...
- 7,119
-1
votes
3
answers
3k
views
New to Data Warehouses/ERP
The organisation I work for is very application rich i.e. there are lots of SQL and Oracle databases. We are thinking about a data warehouse.
I believe there is a difference between an Enterprise ...
- 7,119
4
votes
2
answers
425
views
implementing dynamic query handler on historical data
Context:
I have historical data about property (house) sales collected from various sources in a centralized/cloud data source (assume info collection is handled by a third party)
Planning to develop ...
- 224
2
votes
0
answers
116
views
Best practice or design patterns for retrieval of data for reporting and dashboards in a domain-rich application [duplicate]
First, I want to say this seems to be a neglected question/area, so if this question needs improvement, help me make this a great question that can benefit others!
In my experience, there are two ...
- 3,335
50
votes
11
answers
30k
views
Best practice or design patterns for retrieval of data for reporting and dashboards in a domain-rich application
First, I want to say this seems to be a neglected question/area, so if this question needs improvement, help me make this a great question that can benefit others! I'm looking for advice and help from ...
- 3,335
0
votes
1
answer
481
views
How to provide data to a web server using a data warehouse?
We have data stored in a data warehouse as follows:
Price
Date
Product Name (varchar(25))
We currently only have seven products. Once every business day, seven new data points are added ...
- 285
6
votes
6
answers
11k
views
What are the disadvantages of using stored procedures, SSIS, and SSRS to implement business applications? [closed]
Recently, it was suggested that many projects (where I work) should be implemented using some combination of stored procedures (on SQL Sever / T-SQL), SSIS, and SSRS.
In one specific project, SSIS ...
- 285
3
votes
2
answers
1k
views
Data warehouse architecture for mutating schema
I am setting up an ETL process and small data warehouse for querying the data in a few different dimensions. One issue is that the schema for the objects can mutate over time - mainly that some fields ...
- 231
4
votes
3
answers
6k
views
Can a table be both Fact and Dimension
Ok, I am a newbie and don't really think "dimensionally" yet, I have most of my initial schema roughed out but I keep flipping back and forth on one table.
I have a Contract table and it has a ...
- 162
0
votes
1
answer
1k
views
Developing an analytics's system processing large amounts of data - where to start
Imagine you're writing some sort of Web Analytics system - you're recording raw page hits along with some extra things like tagging cookies etc and then producing stats such as
Which pages got most ...
- 320
5
votes
2
answers
7k
views
What's the best way to learn the MS Business Intelligence stack? [closed]
What's the best/easiest way to get into the MS BI stack on my own (preferably with a [kindle] book)? Specifically, I'd like to learn more about SSIS, SSRS, and SSAS in that order. I've tried looking ...
- 181
0
votes
1
answer
1k
views
Testing BI (Business Intelligence) systems on the MS stack
I'm placing my first steps in the Business Intelligence sector and I'm looking for good ways to test BI systems. I'm going to try to explain what my system consists of so that people unfamiliar with ...
- 1,810
3
votes
3
answers
21k
views
How and when dimensions are filled in a data warehouse schema?
I'm reading Ralph Kimball's books and I'm currently exploring the following data warehouse schema.
Are both dimension & fact tables populated during the data warehouse creation/update? How often? ...
user2567