Hands-on different database management systems

When we all got locked down, I needed a project to keep my brain active. (That gets important as you get older: I just had my 80th birthday).  My wife is learning Spanish and I tried writing a novel (it didn't fly). In the end I decided to do something that I was good at - software development.  I wrote a generic system to update and query a database using node.js. Test system here: http://www.sudsjs.com.

The database is accessed through a driver, which initially was written and tested for MySL, PostgreSQL and SQLite 3. Since then, I have implemented (or tried) various NOSQL systems. So here is a summary and a link to any blog posts about all of the databases I have used.  

NOSQL - a cautionary tale

The sales manager came down to the IT department and looked around. He spotted a young developer who looked a bit bored.  His name plate said 'Luke'. "I have a small job for you if you are interested". Luke  nodded. "We get sales orders from out customers. The warehouse handles them but we can't get access to the data. Can you write a little program to store then on a computer for me."  Luke looked at a file of sample orders. "Sure no problem."

He turned to the jaded old database administrator (Joda) in the corner "This is a great application for a NOSQL Document database like MongoDB or CouchDB."  Joda looked up - "That is a decision, regret you will".  Luke muttered something about 'luddite' and went back to his desk and coded up one NOSQL document per sales order, with customer details and an embedded array of order lines. The logical way to solve the problem.

What was it like?

I learned my software skills on the Ferranti Mercury computer. It had as I recall 468 words of memory (a word is about 6 bytes so call it 3k bytes).   Let's see, my phone has 8,000,000,000 bytes - that's about 2.5 million times more.  My phone makes the 1500 instructions per second look pretty silly as well.

The computer was big. The electronics used valves and they get hot. So each cabinet has a refrigerating unit built-in, and they were noisy. 

I tried CouchDB and I like it

My COVID lockdown project is a system to keep a database updated called SUDSJS. I am using it to check out different database management systems (DBMS).  The latest is CouchDB - and I like it.

Firestore didn't go well for me.

 

I decided to use my lockdown project SUDSJS.com to test out a few different NOSQL database management systems (DBMS).  

SUDSJS is written in JavaScript using the node.js server-side system and I designed it so that I could plug in different drivers for different databases. I wrote database drivers for MySQL, SQLite 3 and PostgreSQL. The first NOSQL system I tried was MongoDB, and the SUDSJS.com test site is currently running using this. The second NOSQL system I tried was Firestore from Google. 

It didn't go well.

Database design - SQL vs NOSQL examples

 

For the last 50 years there has been one and only one way of designing a database.  SQL Database management systems (DBMS) have been the uncontested standard. (A better name is 'relational' because SQL is just the name of the language used to access them, but we will stick with it.)

But things have changed in the last 10-15 years. Databases that disobey relational rules are coming into vogue.  The term NOSQL is used for these non-relational databases (the above chart is work in progress, but I have been looking at these systems.) 

What is the difference between classic databases and this new breed? 

I will use a very simple and familiar example. Students, subjects, exam papers and results.  With the new tools we have new options and I have developed three different solutions to the same problem to illustrate the new capabilities open to us. All three solutions are working on my lockdown project - SUDSjs.com using the MongoDB DBMS.

Handling data consistency in a Document Database

You know what an invoice looks like.  We have all seen thousands of them - probably too many. But invoices illustrate an interesting problem - and I am not talking about your VAT  return. I am talking about the new kids on the database block - document databases.  An example is MongoDB. These can store all the data for one invoice in a single structured record (called a 'document').   

In a traditional Relational Database, the data is held in flat tables, like big spreadsheets. Invoices will be stored in two tables, one for information about the invoice, like customer and date (call it the 'invoice header'). The second table will store all the invoice lines. One row per line. The lines will be linked back to their invoice by an invoice number.  But what happens if the invoice header disappears, but the invoice lines are still there?