Our company leadership is finally getting to the point where they want to use our historical data for analysis and business planning. The problem is, we have no warehouse and have historically (last 2 years) been doing most analysis using our production databases. Now that they want to see certain trend data, I get to be the one to tell them that we don't have the data for it because our systems don't track changes to what they're looking for, only the current value.

One of my personal goals for the next 1-3 years is to stand up a warehouse and ETL layer to make this possible, but I need help in scoping, planning, designing, and implementing. I could theoretically do it all myself, but I'd rather not relearn someone else's lessons. I have an overall idea of what we need and how to do it, but the devil is always in the details.

Does anyone know of any consultants I could talk with to get an idea on platform options, cost, timeline, etc? Anyone with that kind of experience looking for a job?

May help to know location and current technology stack.

I'm a consultant currently doing that for a project.

San Antonio. Basically direct querying our production databases with Power BI, so not much of a real stack.

Do you have a way I could contact you and ask you some questions?

ABATTBQ11 said:

---

Do you have a way I could contact you and ask you some questions?

---

Do you have an email I can contact? I'd say PM me but I don't think you have stars.

Username at gmail. It's a side account, and I'll email you back from my main.

ABATTBQ11 said:

---

Our company leadership is finally getting to the point where they want to use our historical data for analysis and business planning. The problem is, we have no warehouse and have historically (last 2 years) been doing most analysis using our production databases. Now that they want to see certain trend data, I get to be the one to tell them that we don't have the data for it because our systems don't track changes to what they're looking for, only the current value.

One of my personal goals for the next 1-3 years is to stand up a warehouse and ETL layer to make this possible, but I need help in scoping, planning, designing, and implementing. I could theoretically do it all myself, but I'd rather not relearn someone else's lessons. I have an overall idea of what we need and how to do it, but the devil is always in the details.

Does anyone know of any consultants I could talk with to get an idea on platform options, cost, timeline, etc? Anyone with that kind of experience looking for a job?

---

I am not looking for a job, but I will tell you something. We did a

proof-of-concept last year with for a data warehouse. I have been

doing this for about 15 years.

1. Start with a budget.
2. Look at Cloud solutions.
I would suggest Snowflake database
or Microsoft.
3. Enlist a consulting partner.

You didn't say what size company you have. Good luck though.

Thanks. Construction with $600 million in volume per year. At this stage I'm really trying to ballpark costs and timeline. I'm assuming somewhere in 6 figures and that this will be a hard sell to get funded and kicked off.

ABATTBQ11 said:

---

Thanks. Construction with $600 million in volume per year. At this stage I'm really trying to ballpark costs and timeline. I'm assuming somewhere in 6 figures and that this will be a hard sell to get funded and kicked off.

---

I would not try to build it all at one time. List the priorities and add on as funding

and need comes.

They great thing about Snowflake is that computing and storage

are separated. You can upgrade on compute power on the fly. Memory is

relatively inexpensive und virtually unlimited through a cloud platform.

www.snowflake.com

Feel free to send me an email as well if you want to bounce ideas around ( username @gmail.com).

I help lead a Data & Analytics practice for a consulting firm so I am typically doing 3-4 data warehouse projects a year.

My firm has been named partner of the year for Power Bi, Tableau, and Snowflake over the last couple of years...so we have gotten to experience some of the good and the bad of the different platforms.

Best advice I could give is don't get hung up on a "traditional" DW.... Depending on the use cases a well formed data mart can sometimes provide better business value than a DW (especially with a small team).

Thanks. I'll shoot you am email tomorrow.

really sit down and do the TCO on cloud vs on-prem. There are certain applications and workloads that are great for cloud, others are not. Cloud is the "sexy" thing to do, but it can get expensive REAL fast if you put the wrong workload in the cloud. You can always pivot, but its typically messy and expensive to egress from the cloud.