Why you need to think about your database, and when to choose which
Almost every system keeps its important data in a database, and that choice is hard to undo later. A database stores data safely on disk, finds it again quickly, and (for many kinds) keeps it correct when several things change at once. A bad choice or a bad design shows up as slow queries, wrong data or painful changes.
Getting it right gives you two things interviewers always ask about:
- Correct, durable data: what you save stays saved, and related records stay consistent.
- Fast queries at scale: the right structure and indexes keep reads quick as the data grows from thousands to millions of rows.
Choose a relational (SQL) database when:
- Your data has relationships: customers, orders, products that refer to each other.
- You need transactions: several changes that must all happen or none, such as moving money.
- You want to ask flexible questions later with joins, filters and totals.
Choose a document, key-value or other NoSQL store when:
- Records are nested and vary in shape, and you usually read one whole record (a document store).
- You only look things up by one key at very high speed (a key-value store).
- You have a special shape such as a graph of connections, a time series, or a huge write volume that one machine cannot take.
Skip the fancy choice when:
- You are small and unsure: a single relational database such as PostgreSQL is a safe default for most products.
- You pick NoSQL only because it "scales": check what you really need, because modern relational databases scale a long way.
Try it
Choose how the database looks up a row, how big the table is, and what to do. Press play and watch how many pages it reads.
Finding a row with and without an index
Step 1 of 5 · Press Space to play, arrow keys to stepTable scan: rows checked
A table with 1,000 rows. There is no index, so the database must scan the table row by row. Assume about 100 rows fit on a page. Press play.
- rows:
- 1,000
- rows looked at:
- 0
- pages read:
- 0
Teaching model: about 100 rows fit on a page and about 100 keys fit on an index page. Real databases differ, but the shape is the same.
Things to try:
- No index on a table of 1,000,000 rows, Find a row near the end: the database scans most of the table, about 9,000 pages. Switch to With an index: 4 page reads.
- Keep the index and compare table sizes. For 20, 1,000 and 1,000,000 rows, the index needs 1, 2 and 3 levels. A table 50,000 times bigger needs only two more steps.
- Find a row that does not exist with no index: the scan reads every row, 10,000 pages for a million rows. A miss is the most expensive scan.
- Insert a new row with and without an index. The index costs a second write on every insert, because it must stay sorted. Reads are faster and writes are slower.
Types of database
| Type | How data is stored | Good when | Weakness |
|---|---|---|---|
| Relational (SQL) | Tables of rows and columns, linked by keys, queried with SQL | Relationships, transactions, flexible queries | Harder to spread writes across many machines; schema changes need care |
| Document | JSON-like documents, often nested | Records that vary, read as a whole | Joins across documents are awkward; easy to duplicate data |
| Key-value | One key to one value | Caches, sessions, simple lookups at huge speed | You can only ask by key |
| Wide-column | Rows grouped by a key, with flexible columns, spread across many machines | Very large write volumes, known access patterns | You must design tables around queries; limited ad hoc queries |
| Graph | Nodes and relationships | Questions about connections: friends of friends, routes | A niche; not the best for plain tables |
Other special-purpose stores exist, for example time-series databases for measurements over time and search engines for text search.
Normalised vs denormalised schemas
- Normalised: each fact is stored once, and other tables refer to it by key. A customer's address lives in the
customerstable, and orders point at the customer. Updates are simple and there are no copies to disagree. Reading an order and its customer needs a join. - Denormalised: facts are copied to where they are read, for example the customer's name stored inside every order document. Reads are cheaper and need no join. But when the name changes, every copy must be updated.
Start normalised. Denormalise on purpose, for a measured read problem, and know how you will keep the copies in sync. Document stores lean denormalised by design.
How an index works
An index is a separate, sorted structure that maps a column's values to where the rows are, usually a B-tree. Without it the database must check rows one by one (a scan). With it, the database walks a short tree of pages and jumps straight to the row.
- The query says
WHERE customer_id = 42. - The database reads the index root page, which says which child page covers 42.
- It follows one page per level, a few reads even for millions of rows.
- The last index page holds a pointer to the table page with the row, and the database reads it.
Rules to remember:
- Primary keys and
UNIQUEcolumns are indexed automatically in most databases. - Index the columns you filter, join or sort on, such as foreign keys.
- An index has a cost: extra disk space, and an extra write on every insert, update or delete. Do not index everything.
- A composite index on
(a, b)helps queries ona, or onaandb. It does not help a query onbalone, so column order matters. - Check with
EXPLAIN. It shows whether the database really uses the index.
Relational vs document vs key-value
| Concern | Relational (SQL) | Document | Key-value |
|---|---|---|---|
| Relationships and joins | Strong | Weak, usually copied | None |
| Transactions across records | Strong | Limited, varies by product | Usually single key only |
| Flexible shape | Needs a schema change | Easy | Easy |
| Flexible ad hoc queries | Yes | Some | No, by key only |
| Raw lookup speed | Good | Good | Excellent |
| Best for | Orders, accounts, anything related | Profiles, catalogues, content | Caches, sessions, counters |
A relational database is the safe default. Use a document store when the record is naturally one nested object, and a key-value store when the question is always "give me the value for this key".
Avoiding a new single point of failure
The database is usually the most important single point of failure, because every app server depends on it. Protect it:
- Backups, and test restoring them. A backup you have never restored is a hope, not a plan.
- Replication to a standby that can take over (see the Replication and Sharding lesson).
- Limits on connections and slow queries, so one bad query cannot take everyone down.
- Do not share one database between unrelated services, or one team's bad query becomes everyone's outage.
The trade-off: more copies and more machines to run, in return for survival after a failure.
Common mistakes
- No index on a column you filter or join on. The query scans the table and slows down as data grows. Add the index and check with
EXPLAIN. - Too many indexes. Each one slows every write. Keep only the ones your queries use.
- Storing money as floating point. It loses precision. Use whole cents or a decimal type.
- The N+1 query problem. One query for a list, then one more query per item. Fetch them together with a join or a batch.
- Picking NoSQL for the label. If your data is relational, you will rebuild joins by hand. Start from your access patterns.
- Never testing restores. Backups that fail on the day you need them are common. Practise restoring.
How to talk about databases in an interview
- Start from the access patterns: "the main queries are these three", then pick the store and the shape of the data.
- Say why SQL or NoSQL, using relationships, transactions and query flexibility, not fashion.
- Name the indexes you would add and what they cost on writes.
- Mention unprompted: backups, replication, and what happens when the database is slow or down.
- Name the trade-off: for example, denormalising for fast reads at the cost of keeping copies in sync.