Let's face it, there is a wide variety of skill out there. If you spend any time monitoring Stack Overflow, MSDN forums, Ask SQL Server Central or twitter's #sqlhelp hash tag, you'll be very well aware that there are people out there barely managing their database environments – people that some of us wouldn't let within a mile of our own data centers. But a lot of these people are not trying to build the next facebook or amazon – they're just trying to keep their small business running, and don't necessarily need perfection.
One of the "live by the sword, die by the sword" problems with SQL Server is that it has become so easy to manage. (I remember one of the mantras of SQL Server 7.0 was to make it so self-tuning that a DBA wouldn't be needed. That's not quite how it turned out, but a lot of systems are certainly less hands-on than they were in the 6.5 days.) Because of this, in probably a good 80-90% of cases, a business can get by with minimal or no DBA expertise. While some of the things that people will set up instinctively (e.g. maintenance plans) will not be the most efficient things in the world, they're also not going to be bad enough to cause the business any grief – they're just not trying to squeeze so much performance out of the system that these minor inadequacies are really going to cause them any grief.
Does that mean you should stop learning, and ignore great blog posts on backups, compression, security, and other important aspects of SQL Server? Of course not. The challenge is to know when you will need those skills to be present. At the beginning, for most start-ups, I wouldn't suggest that a rockstar DBA is priority #1. But as your application becomes the next big thing, there will come a time when boot-strapping it just isn't going to cut it anymore. If you don't have a database person on staff or retainer periodically looking at your stuff, you can end up in a really bad situation.
A lot of the things that you started on an "expertise budget," we'll call it, can be fixed later on; but some things will be more challenging. Storage, for example, is important… but as long as you can afford maintenance windows or other scheduled downtime, moving from DAS to SAN is something you can do when it's required (and when the budget is justified). Similarly, scaling up to larger machines, or adding memory, are things that are relatively easy to do and don't necessarily require a DBA (except maybe in downtime planning or data migration).
On the other hand, choosing whether to use IDENTITY or application-generated GUIDs as a primary key is something you should put more thought into up front, because re-engineering both the schema and the application(s) is not something you can pull off during a Saturday afternoon maintenance window. Nor is being prepared for scale in general… you don't want to get caught with your pants down, thinking 10GB drives were big enough, and you run out of disk space on day 2. You probably don't need a high-end DBA for ballpark capacity planning (take your best guess and double it), or to figure out efficient ways to replicate and distribute data.
Now, please don't take this the wrong way. I am not saying that DBAs are useless, nor am I suggesting that you *should* avoid a DBA at all costs. I just think that, in some scenarios, you can live without a DBA – for a little while. Not all applications need to be bleeding edge, and many will never grow beyond very meager needs (I wonder how many of today's apps could feasibly run on Express?). In fact many of these apps are going to start moving to the cloud, where even the rockstar DBAs out there won't have much control over storage, availability, redundancy, etc. So some DBA skills are, at least to some degree, being deprecated right out from under us. Maybe you're not currently in a position where you need to worry about this, but how confident are you that this will always be the case?
Thanks to Paul Randal (blog | twitter) for hosting and coming up with this month's T-SQL Tuesday topic!
I am with Aaron in this discussion. In many cases all we need from the database is to persist objects. SQL 2000 + some basic database skills is good enough in many cases, anything more than that is an unnecessary complication that just uses up precious resources and prevents us from getting things done.
Armando,
I think you are observing survivor's bias – you probably mostly see the startups that made it to profitability and can afford a consultant. You don't see the ones that failed, and most of startups do fail. Can anyone estimate how many failures are caused by overspending on too much design, but trying to do it according to a theoretical textbook?
Grant,
I am not with you on stability. Right now once a day one of several apparently identical app servers switches off daylight savings time; the rest run on correct time. Someone in our team had to research, find some KB, and apply a fix. We have still to see if it worked. As we are applying patches, we are not doing anything more useful, so we are losing money.
As Ray Ozzie said: "Complexity sucks the life out of users, developers and IT. Complexity makes products difficult to plan, build, test and use. Complexity introduces security challenges. Complexity causes administrator frustration."
Isn't that an overly simplistic example? I'll go back to the experience well. I did work at a silicon chip broker who had a total staff of three people who did nothing but data entry. Their system was riddled with deadlocks and blocking. They had no IT person (which is why I was there). The original architect as a VB developer who loved loops and who obviously never heard of a indexes.
Sure Armando, I've run into the same kinds of issues over the years. While those "solutions" are dirty and not the most efficient, they do get the job done, and the problems with them are not going to affect the majority of small shops, ever. In some cases they would be affected, but if it takes Mary in accounting 5 seconds to run a report that *could* be run in 1 or 2 seconds, having a DBA come in to help improve the normalization of a table is not likely something that will be cost justified. When the report runs forever, or returns the wrong results, that's a different issue (and those problems don't necessarily require an expert to solve, either).
There probably are times when a small shop needs some specialized help, but a lot of them really can run for years, even decades, on their small systems without any issues, just because those systems are small, and because, let's face it, SQL Server and Windows is a lot more robust than people generally give it credit for.\
Believe me, I want to keep my job, and have jobs into the future, but I can see why small shops just do without DBAs.
I'm not necessarily talking about SQL rock stars. I'm talking about having someone competent that can guide or train a small or start up company in making the right technical decisions up front.
I consulted for a time. My experience has shown me that the companies that have developers that know the no-nos are not necessarily the rule. Most that I've run into don't care about the database itself; they just want to stuff the data in and get the data back. Not many cared about the details (that was up to me). One company I was at, we had problems with non-prepared queries being issued that caused memory problems. At another, most tables were in 1NF with all kinds of parsing routines to get at data. Yet another would pull in entire tables only to loop and throw away the rows they didn't want. In fact, if you were to examine some of the application coding examples that are shown at the University level, you'd see in-line SQL and loops used as examples in teaching data access.
Armando, I think there's a big difference between knowing not to code recordset loops in Java and being a SQL Server rockstar. I think in a lot of cases small shops can do just fine with developers that have enough exposure to databases that they know the big no-nos (even if they don't necessarily understand why they're no-nos).
I have to respectfully disagree with you, Grant. I think the small shop or start up should at least have a part time consultant at a minimum. Too many of these types of companies (and I've seen this first hand) think of the database layer as a storage container. The data is rarely well normalized (or thoughtfully denormalized) and, usually, not every piece of data required is stored in the database. For instance, one company I consulted at was generating reports using loops embedded within their Java code while also going out to an XML file to get additional pieces of data. My belief is data is the most critical asset of every company. I believe it should be treated as such.
On a side note… My dad built houses for a living and was an electrician by trade. He was handy in all other aspects of construction but would sub-contract out the non-electrical work. When I asked him why he didn't save some money in some areas and do it himself (like plumbing), he said, "Son, I wouldn't hire a plumber to wire my house".
I have to agree. The small shop & the startup really can do without. But once that line is crossed, Katy bar the door.