It has been a long journey getting through this book, 8 weeks actually. I can honestly say I have learned a heck of a lot and know I can be more effective in the future. For the last couple of years I have felt like I have had a giant gaping hole in my knowledge when it came to different types of databases, and this book has really helped me start to fill in that hole.

Impressions of the Book

While the book has strong and weak parts, overall it did a great job at giving just enough information for you to learn each database. The only negative of the book is it isn’t longer, more detailed, and doesn’t have more in-depth examples of data to store, but that is because I like details to be extensive as possible even if they are too much. The book accomplished its goal well and was surprisingly short based on everything I learned.

What I learned

Day 3 of Redis Chapter

I hinted at really liking Day 3 of the Redis chapter, and it making me a better developer. What I was not telling you is that Day 3 was all about using 3 separate databases Redis, CouchDB, and Neo4j to build a single application. It was all built on Node.js so it can give you a real look at the breadth of technology it can be used with.

What was very interesting is how it was used in conjunction with everything, and it was well layed out showing yet another way to use the databases. I’ll be honest it did not go into as much detail as I wish it would have, but it did just enough to drive the point home, which was the point.

Types of NoSQL Databases

I never realized there were so many different types of NoSQL databases. Part of that was because of everyone calls it NoSQL, and the other part of it was because I never looked beyond MongoDB. The greatest thing about reading this book is I am not only aware of the different types of databases, but I am starting to understand when a good scenario is to use them in. It gives me a better leg up for the future when I might run into scenarios to need them since NoSQL is becoming more important.

To the Future

MongoDB

For the immediate future I am going to start off on a project and use MongoDB. I am choosing it because I feel the most comfortable with it, and I am not sure which type of NoSQL store I should use. I do however know for this project NoSQL is the optimal data store over an RDBMS, which is a win in and of itself. I kind of feel like I should use a key value pair store, but I figure since this is a side project for fun if I choose wrong it really doesn’t matter since it will be fun and I will learn.

Neo4j

Believe it or not by the end of the year I will start in on using Neo4j for another project, and I will bring you along for the ride. I have had an idea for an application for a long time to help with my learning Japanese. The problem is how I envision the project and its complexity I have always had trouble with the database schema. After working with Neo4j it hit me that it might be the correct fit for my specific needs, maybe. I will be experimenting with it and using it with Node.js as well.

Conclusion

I find the fact that I am now able to analyze the needs of my project and start the process of choosing a data-store in an informed manner to be a huge win for me. I walked in with no knowledge and a lot of confusion, and have emerged with a better understanding of how to solve some of my problems with a couple of projects. Problems I didn’t know exactly what they were or how to solve them before, I only “felt” the problem.

I do recommend this book to anyone interested in learning more, and about, databases. To be honest it was a bit much to try and do each chapter every week, and try to understand it fully. If you are at least a little bit familiar with NoSQL DB’s you should be fine, but if you were like me and had no understanding at all then take more time. Other than that get the book, learn and become a better developer so you can make even cooler stuff.

The best was saved for last, at least that is how I see it, and in multiple ways. I will be honest I was a bit surprised Redis was a Key Value pair based data store, but I guess I should not have been considering its efficiency. As the book is winding down I wasn’t sure how things were going to end, but they are ending with a big bang, creation of the universe kind.

Impressions of the Book

This chapter was actually a short chapter if you don’t include day 3, and I won’t. Redis is surprisingly easy to learn and use, and the authors did a good job building up from the basics all the way to fairly advanced usage. Day 3 was probably the best part, but I don’t want to talk about it this time. All of the examples were very good and fit just right as building blocks to a better understanding.

What I learned

Data Structure Server

One of the ideas behind Redis is that it is Data Structure Server. Meaning it can store all sorts of data as different types of Data Structures. When I started reading about it I was a bit confused as to what you could even do with storing data as multiple types of data structures.

The example they used, I think, fit so well I dare say it is the best example of the entire book. Their first example was to use Redis as a back-end for a URL Shortening service. The authors did a very good job of showing you how to build the entire back-end to this service on Redis. They did a good enough job using the different types of data structures you could go and build your own service to compete with TinyURL and Bitly.

CLI and *nix

Redis feels very *nix’y™ with its focus on the command line, and simple configuration ability. Day two does a good job at introducing you to administering Redis instead of, you as the user, trying to filter through tons of documentation and hoping you read the right thing first. The book does just enough to get you comfortable using Redis on the command line and doing configuration you are eager to explore.

Grease

The best quote from the book that really grabbed me was this:

Redis is like grease.

The best part about reading the book and reading up on Redis is how it helps, and acts as grease in the gears to keep things going smoothly in your applications. I never understood how you could do that with a separate data store until I read this chapter.

Overall

Redis seems to be a very flexible data store with a lot going for it, especially since it has an amazing amount of speed to it. With it being popular among a lot of companies it has been battle tested in the wild and seems to shine when used properly. To be honest after reading this chapter I think Redis and MongoDB are the two NoSQL Databases I can understand how to use the best.

Next week is the final post in this series of blog posts where I will do a final review of the entire book, the last chapter, and also talk about, perhaps, the best part of the Redis chapter, Day 3. It really makes the entire book worth the read, and I feel has already made me a better developer.

What a mind bender, especially when structure is all your brain is used to. I really never thought too deeply about the idea of a graph based databases. However, why not? We have other types so a graph based database seems inevitable. The question is how am I going to understand it?

Impressions of the Book

This chapter was did well with the examples. Based on what I was given everything made sense so it was a good chapter on that basis. It was a thorough overview with a lot of good specifics in the right places most of the time.

What I learned

To be completely honest I am not really sure I learned much beyond what Wikipedia could offer me, but it is completely my fault/problem. As mentioned in earlier posts my brain is fairly well wired for rdbms’ and the Key Value pair and Document based NoSQL Db’s have really caused me some problems. Having a graph based one has really done a number on me.

I Don’t “Get It”

One of my biggest blockers was; outside of the example data given I really couldn’t expand to any other types of data to be stored. Normally I can come up with something to better understand the storage strategy, even it if is bizarre. I think Graph Based is just a bit beyond me right now, but I am going to stick with it because it seems very promising in what examples were given.

Scale Up and Down

I was happy to see that Neo4J was able to scale up and down. It can even go from 1 to 34 billion nodes, that is impressive, and allows for some real flexibility.

Overall

I am going to keep researching on Neo4J since something about it is just very interesting. Granted I really don’t understand it too well right now, but that doesn’t mean I won’t in the future. If you can figure out how to use it well it seems very promising. Also, if you “get it” please come back and try to explain it to me better.

The final chapter is coming, and I bet they are saving one of the best for last, maybe. Next week is Redis, and I peeked at the first sentence of the chapter and it is very intriguing, I will let you wait until next week to find out what it is. Redis gets a lot of buzz for various reasons so I am excited to read about it.

“If I knew then what I know now” is what comes to my mind after reading this chapter. CouchDB was the first NoSQL DB I tried, and it made very little sense at all so I gave up using it. However, I wish I had read this chapter when I first started so things would have made more sense.

Impressions of the Book

This chapter was a tough read because the flow of the chapter did not go well with my brain. That said this was the last chapter they finished writing so it will probably be edited. I am not sure the data scenarios worked well either, but that is more of a problem with me than the book.

What I learned

Things are starting to get repetitive on “what I learned” since it seems all NoSQL databases have some of the same goals in mind. However, just like the other databases there are things I took away which I like and/or stood out to me on the simplicity.

Revisions

Something which always has bothered me about Databases is the locking while data is being written. While I have never had any problems with this, ever, the idea of it has bugged me. CouchDB does not do any locking it just writes concurrently, but sets a revision field. Each request needs this revision field to make sure something has not been overwritten since you last pulled the information. The best part is the revision system is easy to understand and work with.

Views

I was very happy to see you can do views in CouchDB, my original problem was how to get data out in a good organized fashion for use. I always had some hit or miss issues when getting data, so it is great to have views available.

Size Doesn’t Matter

One of the best features I liked about CouchDB is it can scale up and down fairly well. That means you can even run it efficiently on an android device if you needed to. I have not heard of any others being able to do something like that so it is an interesting use case.

Overall

I now have more understanding of CouchDB than I used too, but I am not sure when I would use it over something like MongoDB for web development, which is what I do. Couch seems very terse and to the point, which I guess is a benefit. This one I am going to have to leave up to other people to help me figure out if I need it or not.

Next week is the Neo4J chapter. This one shall be very interesting to read since it is a graph based NoSQL database. I am very interested to find out how it works.

MongoDB seems to be the database that everyone flocks to when they are starting out with a NoSQL Database, and for good reason. This week was a bit more interesting going through the book as I had to look a lot of stuff up on my own since a few assumptions were made by the authors. It all worked out though.

Impressions of the Book

This chapter seemed a bit terse and shy on information. Oh there was a lot of good information exchange done in this chapter, but I felt it could have done more, specifically with a few more examples on querying data based on scenarios. Other than that the chapter was much like the rest a lot of information, not a lot of fluff.

What I learned

I spent a lot of time doing examples to be sure I really understood them. Also compared to the other NoSQL databases there are a couple of things I liked they didn’t have. I am going to avoid being redundant so I will not expand on how easy it is to scale MongoDB as well.

Querying Data

Unlike previous NoSQL Databases you can actually do some really cool queries on the data to pull back specific information you need. There are some great mechanisms in place to do just this. I really liked there was a couple of reasonable scenarios to use these queries on.

Indexes

One of the biggest surprises I had was that MongoDB has indexes automatically, and ones you can write as well. I have always read the propaganda about NoSQL it generally talks about how fast NoSQL is so it doesn’t require as much tuning, if any at all. I guess it is relative compared to RDBMS’s. Anyway, I was pleasantly surprised at the speed of MongoDB out of the box, and even more so once you start using indexes.

Overall

I can definitely understand why people jump to MongoDB as their first NoSQL database. With a strong query language and great support in ORMs it is easy to get started with and, almost, not notice it is there. I am definitely going to play with it some more, and try out a couple of scenarios I would like to throw at it. It was an exciting chapter to get through as pieces are coming together in my head to understand more and more of the NoSQL world.

Another week down, and a few more to go. Next week we will jump into CouchDB. I shall be interested in this one because I tried to use it once and failed miserably. Part of the problem was not understanding map reduce, and the other was my lack of knowledge of Document databases. We shall see how things go.

We are on the second NoSQL database of the book. I had personally never heard of HBase before this, but it is an impressive Database. It also provides a good look at different programming techniques and stacks. Fortunately, HBase is fairly simple to get introduced to.

Impressions of the Book

The humor in the book is still going strong and is well used to prove their points. My favorite from this chapter is, roughly, “If you have less than 5 nodes you’re doing it wrong”. This was another good chapter, and I was ecstatic about the example they used of what type of data they were storing because it worked well for a NoSQL database.

What I learned

This was a difficult chapter for me because I wasn’t able to get HBase installed to do the exercises. It got to a point where I was spending so much time trying to get it installed I wasn’t reading the book, so I made sure to read the book. That doesn’t mean I didn’t learn so here are a few key things that stuck out in my head.

Clustering

The thing I am liking as I go through these last 2 chapters is how easy it really is to cluster together servers for horizontal scaling. This has always been one of those things I have never needed to do so I don’t know much about it. It is very well built into HBase by default.

Java and JRuby

This has less to do with HBase than other stuff, but I tend to avoid Java altogether. It seems though that Java is a central part to interacting with HBase. The cool thing was I go to see some JRuby in action. Hopefully in the future if I need to write some Java I can use JRuby because it really is well written and makes interacting with HBase that much simpler.

Scaling

The final part of the chapter, Day 3, was about scaling out HBase. Several things were discussed with regards to apache incubator software built to auto scale out, many things, including HBase. Even went so far as to show you how to scale to many EC2 instances with very little work. Seeing how easy it is, is starting to give me a bit of confidence in doing it in the future if I ever need to.

Overall

This was a cool database built specifically for “Big Data”. It was a little hard to associate much with it because I haven’t had the need to do anything with large amounts of data, especially in the upper 100′s of GB and TB levels. However, I am aware of it now so I can pay attention if I ever deal with that amount of data.

I was also ecstatic they dealt with wikipedia data because it was a great fit for NoSQL data because the relationships aren’t all that important. Each article is a single silo of data which is independent except an arbitrary link to another article. It is getting exciting about how much better things are going especially on types of data to store.

That wraps it up for HBase review, next week is the most popular NoSQL database in the Ruby community, from what I have seen, MongoDB. It shall be very interesting to see how it goes. The cool thing is its already installed on my machine from previous attempts to learn it.

Jumping into the first foray of NoSQL Riak was a bit daunting, but thanks to the guidance of some great authors I came out the other end of the chapter relatively unscathed, and maybe a bit wiser. Of all the NoSQL databases the Key Value store seemed the most useless to me, well I was wrong.

Impressions of the Book

Based on the PostgreSQL chapter I was not really sure what to expect with this chapter. My knowledge of NoSQL at best has been conceptual in that you don’t use sql, deep thoughts right. Seeing as how you kind of needed to know what was going on with RDBMS’s for the PostgreSQL chapter I was nervous.

Fortunately the authors anticipated this problem. It seems there goal was to introduce the reader to a solid Database which was basic in concept, but deep in ability so as to ease the reader into the NoSQL waters. I think Riak is a great database to do just that, and the authors did well at introducing and conveying the “how” of using a NoSQL database.

What I learned

A LOT. If I wrote about everything I learned about you would be here for a while. I am still wrestling with some concepts, but overall I am better equipped to understand what is going happening. So I will break down the most significant things “I” learned. First, though, I will bullet point a list of several things which were discussed.

• Link Walking
• Data Partitioning
• Horizontal Scaling
• MapReduce
• Eventual Consistency
• Extending Riak

Riak Reminds Me of S3

They even mention in the book that there are some similarities between Amazon S3 and Riak, which is a bit of an understatement. So with S3 in mind I progressed through the chapter and with everything framed in that way it made much more sense, faster.

Everything is partitioned into buckets and anything can live in those buckets from text to binary objects. So the organization is very similar as well. However, the thing it does do that S3 doesn’t is allowing you to link key value pairs, which brings us to the next thing.

Link Walking

One of my biggest barriers to entry into NoSQL databases has been how you retrieve data. While straight storing large amounts of data is needed you need to actually cross reference the data in some way. Most of my research hasn’t actually talked much about this or how to accomplish the same thing in similar ways.

In Riak you can associate key value pairs, but only in one direction. You can associate n to x, but x will never know it is referenced, at all. Knowing this you have to be aware of how you are going to pull your data out if you choose to use link walking, which is traversing over links from record to record.

MapReduce

I have seen hints about MapReduce a lot of different places, but never looked it up or had a need to use it. This chapter has an introduction to it which really drives home another way to get information out of a data store. It is a bit of a mind bender at first, but has started to make more sense, kind of. It also seems that it is going to be used a lot more in the book for other databases.

A quick idea of how it works is actually based in its name. First you map data together then you reduce them to a single piece of data you need. The interesting thing is it runs on the server so your application itself is not handling the processing of all the data. Another cool caveat is you can save your MapReduce functions as text in a Riak bucket and call it later, kind of like a stored procedure.

Scalability

Based on how Riak was designed it is super easy to add multiple Riak servers together and have data spread across them all so you definitely have some failover. The multiple servers are put together in what is called the Riak Ring, and everything is divided out across them as virtual nodes. As you write to one server data will eventually be written to the others. Yes, there are some things in place for data collision resolution.

Overall

I now have a better understanding of how Riak works, and if I have simple data to store while needing flexibility in my clustering it will be my choice. The book does not really explain well/much when to use Riak except for anecdotal evidence so it is still up to the reader to come up with that on their own. The examples they give work and work well, but I am not convinced a relational database wouldn’t have been a better choice.

All that being said it was a solid chapter overall, and I took a step further in my understanding of NoSQL so the future is looking bright. The next chapter is on HBase, which I have never heard of, so it shall be interesting to see how things go.

Relational Databases are great databases, and a good staple of any development stack when flexible querying ability is needed. Currently my main RDBMS is MySQL, but since the acquisition by Oracle I have been a bit turned off by it. Something always rubbed the wrong way as it was, but the purchase made me really want to move away. However, I never knew to what, or what to read to decide. This first chapter sealed the deal on my move to PostgreSQL for all non-wordpress and future projects.

Impressions of the Book

As noted above knowing what to read about PostgreSQL has been hard, but the book nailed what I wanted to know. I don’t really care too much about SQL Syntax, but what I wanted was more of the nitty gritty of how it worked along with what array of abilities it has. I got that and then some.

While some of the content was over my head because I have never had to deal with that kind of data storage and retrieval I could really see the flexibility incase I need to go there. The best part of the chapter is it was a deep dive with all the assumptions that I knew what I was doing with an RDBMS which allowed the content to be more specific.

What I Learned

For this database this section is going to be a bit small because I am used to RDBMS’s, but that doesn’t mean I didn’t learn anything. There are 3 key things that suck out in my head the most which I like, but are probably trivial to many other people. These might be in other database engines, I don’t know.

Sub Selection

I had never thought about this before, but this is a great way to keep referential integrity without having to lookup id’s everywhere. Here is the code that drew my attention to it.

INSERT INTO events (title, starts, ends, venue_id)
VALUES ('Mobi', '2012-02-06 21:00', '2012-02-06 23:00', (
     SELECT venue_id FROM venues WHERE name = 'Crystal Ballroom'
)

LIKEs

I specifically liked ILIKE which is case insensitive like vs a case sensitive LIKE. I have not seen this before and I remember running into an issue in the past, not sure what RDBMS, where I wanted all uppercase only, but kept getting mixed results. It is cool there is a distinction.

Extensibility

To be honest I had never thought about writing, or having, extra functionality in a database engine before from 3rd parties. This chapter not only showed me you can, but there is a lot of power in using them.

Overall

Chapter 1 was a great read, easy to follow and witty. Good amount of information in a short amount of time with not much fluff. It was a fairly deep dive as advertised for one reasonably length’d chapter. I really liked in the wrap-up where they talked about the strengths and weaknesses of PostgreSQL. However, I can only hope they expand on what they said in the weakness section in later chapters by detailing out how to use non-relational data.

I have high hopes for the future chapters as I was quite impressed by this first one. Join me next time for Riak.

Databases are annoying and generally suck. Many us developers prefer to just do stuff with data and don’t like to deal with storing data beyond a nice pretty api. However, to be an effective software developer we also understand knowing about databases is a smart idea, doesn’t mean we have to like it.

One of the biggest problems for me is I don’t really know a lot about data stores outside of RDBMS’s. NoSQL data stores do not generally make a lot of sense to me, I know about them technically, but I haven’t found (m)any good overviews which help things click on proper ways to use them. It is my hope that reading 7 Databases in 7 Weeks will be that final piece of knowledge which helps everything click into place.

A Note

I feel I should note before moving on this book is still in beta so things can, and will probably change. Most of the book covers different NoSQL databases so with my limited knowledge anything is better than nothing. I just wanted to be sure to point out as of now the book is in beta, but that doesn’t mean it is wrong or bad.

What to Expect

Over the course of the next 7 weeks I will take one database at a time and do all the exercises, to the best of my ability, and read the entire chapter. I will then describe what I have learned and what, if anything, really helped things click in my head. Finally, on week 8 I will do a wrap-up of everything I have read and my opinion of the book.

What is In the Book

Databases of course . The book is broken up into the following databases:

1. PostgreSQL
2. Riak
3. HBase
4. MongoDB
5. CouchDB
6. Neo4J
7. Redis

Each database will get its own post so I can do them justice. Especially since there are several different types of databases.

What I Hope to Get Out of The Book

I mostly use RDBMS’s, namely mysql, but I need to learn NoSQL, and want to learn PostgreSQL, if for nothing else to know when and where to use it. I don’t want to just use it because I can, but because I should. In the first chapter, which was an overview, this is what they promise and I think they will deliver. I don’t intend to be an expert in each database, but at least have a working knowledge of them to combat my database ignorance.

Please comeback next week for the first part on PostgreSQL.

You see a lot about SQLite all over the place, but finding a good place to start with how to use and interact with it is a pain. For a long time I never understood how to create, update, edit, etc. a sqlite Database. It turned out to be easier than I thought. Therefore this a quick get started below. One thing to note is to remember that SQLite is a subset of SQL* most basic commands will work, but you might need to look up the nuances.

Download SQLite

If you are on linux or OS X you might already have it. Open up command line and try to type:

sqlite3 –version

If you get a version number you are good, jump to next step.
If not you need to visit http://sqlite.org/download.html and get your relevent version. For windows get the “Precompiled Binaries”.
If you want can go ahead and extract the executable and add the .exe to your path, or just run command-line from that folder.

Create a Database

Creating a SQLite database was always the most confusing thing for me what to do, but I found it was pretty simple. If you have SQLite executable installed you simply type:

sqlite3.exe test.db

This will create the db file and get you into console mode ready to edit the database. Usually I like to try to find a GUI to edit the db, but for some reason I can’t find a very nice SQLite GUI that will create a db file so have had to do it manually. Unfortunately, when I try to create the db file if I don’t create a table before I exit it doesn’t actually create and save the file for use; so I usually create a small little table.

Create Tables

This is actually really simple since it is ANSI SQL

create table test_table(id INTEGER NOT NULL, name TEXT);

“Don’t forget the semi-colon at the end!”

See Schema

Use the command line tool to view the design/layout of your database. Use “sqlite3.exe db_filename.extension” on an existing SQLite db file and you are ready to go editing and viewing the file.

.schema

This command actually outputs the commands issued for the created tables. So it will look just like the command you input above. This is how you will view your database structure.

Exit and Save

FInally comes time to quit. This is quite simple.

.exit

Now you shouldn’t be stuck trying to close the file for 5 minutes because you don’t know what is going on like I did with the stupid period before the word “exit”.

Conclusion

SQLite is a really useful and portable database file to use, and was pretty “scary” to figure out when I didn’t know anything about it at all. I was so confused by it I didn’t want to mess with it, or anything relating to it. However, considering more and more things I work on seem to be dealing with SQLite db’s it was time to learn it, most notably are RoR and iPhone/OS development. I also like that it works with NHibernate if I needed.

I think if this is all you ever learn about SQLite then this is enough to make you comfortable with SQLite when you run across it in the future.

*As noted by Jay R. Wren in the comments. I originally said it was based on T-SQL. This was my confusion because every one I talk to about SQL always says T-SQL or SQL. I never knew the history behind it. SQLite technically uses a subset of ANSI SQL which was formalized by ANSI in 1986, aka SQL-86/7, it has progressed since then and T-SQL is a Microsoft SQL Server and Sybase variant of SQL, sorry for any confusion on this. For more information feel free to read the wiki article on SQL. SQL – Wikipedia