Eric Florenzano's Blog - Latest Comments in My Thoughts on NoSQL

Re: My Thoughts on NoSQL

Koowie — Thu, 18 Aug 2011 10:49:59 -0000

This is a good article. Koowie.com is in a mysql database. It fits us operfectly fine. But if we get big I wonder if it is worth moving to NoSQL.

Re: My Thoughts on NoSQL

aleksny78 — Sat, 04 Jun 2011 14:02:24 -0000

Well, what do I think, NoSql rules, however, regular SQL rules as well :) And, yeah, agreed, for 99.9% of sites regular SQL is enough.
Nowadays, we can see some massive NoSql movement, but, I suppose, it's temporary, as soon as folks moving to NoSql will understand what's exactly NoSql is(not only cool web-scale stuff but also some limitations such as lack of transactions, etc.) it's popularity will be decreased a little and it'll took right place in technologies stack.

BTW, I've attached an image describing life-cycle of some new technology. I believe, NoSql is a bit after "Peak of inflated expectations."

Cheers :)

Re: My Thoughts on NoSQL

Blue Heeler — Mon, 28 Feb 2011 04:03:35 -0000

yes.that is a great Thoughts on NoSQL.Post agree that 90% off sites are good with SQL. I 'm in addition not too impressed with thinking about moving all the images manipulation into the work layer from a credit card applicatoin scalability perspective. I believe you hit the nail to the head with your own comment "Most men and women... relying less to the features provided through traditional relational sources and engineering a lot more database logic of their application code. " Is it possible go in to be able to more detail as well as provide some hyperlinks about why kitchen table mode sucks by using Tokyo Cabinet? I'm looking within this for any project and it might be very helpful to learn any drawbacks before it gets into to production.

Re: My Thoughts on NoSQL

remove Antivirus .NET — Sun, 27 Feb 2011 03:42:43 -0000

Its Pleasure to understand your weblog.The above articles is incredibly amazing, and I truly enjoyed reading your blog and points which you expressed.

Re: My Thoughts on NoSQL

tyddyni — Mon, 29 Nov 2010 07:04:56 -0000

Don't forget the huge shift to outsourcing. This means that business and government increasingly rely on processing tag-value transactions from their trading partners. Mapping these into RDBMS can at best be complex at worst be impossible without major enhancements to the RDBMS schema. Tag-value data is simple to map on a one for one basis (there are many SaaS services for this) and can readily be processed on non-relational Datastores with minor rule changes

Re: My Thoughts on NoSQL

Vivi An — Tue, 24 Aug 2010 00:22:14 -0000

good news, i like it
http://www.makeyoutubevideo...

Re: My Thoughts on NoSQL

leebert — Fri, 20 Aug 2010 22:54:01 -0000

About the Perl Sql::Statement module ... there are Perl modules for TC/TT, don't know if they are qualifying DBI interfaces that'll work with Sql::Statement. It'd be interesting to see how a Perl SQL::Statement-TC/TT stack would perform vs. a similar integration of Berkeley DB with Oracle's new SQLite front end, also using SQL::Statement (or even libdrizzle, SQL4sockets or SQLite-Server).

Re: My Thoughts on NoSQL

leebert — Fri, 20 Aug 2010 22:44:09 -0000

There's always the Perl SQL::Statement DBI module....

http://search.cpan.org/~reh...

Re: My Thoughts on NoSQL

leebert — Fri, 20 Aug 2010 22:43:25 -0000

If you can stand using Perl, there's the SQL::Statement DB module

http://search.cpan.org/~reh...

Re: My Thoughts on NoSQL

Ben — Thu, 03 Jun 2010 15:20:11 -0000

It's all about what you care about.

For most business applications, things like ACID compliance, a fixed schema with constraints and joins are ahem, pretty nice things to have. Sure make your life easier as a coder and make for more robust applications.

A lot of people whine about sql, which to me is pretty strange... I've never worked in a language so easy and well suited for its purpose. I think the lack of durability of most alternatives compared with sql is a pretty telling indicator. ORMS can be good, but I look at them mostly as a convenience feature and a a way of making schema changes easier. I think the value they provide as an abstraction varies widely... improper use makes things much more confusing. In any case, in most systems they are leaky abstractions because for the best possible performance you have to chuck them and do some things with sql (or the ORM's pass through equivalent). Once you do waive the white flag and make this decision, it's usually surprisingly easy to get the job done. I think most people who hate sql just come to this point so rarely they've forgotten the language in the meantime.

It's true that to get best performance you have to relax ACID somewhat, and there are times when the solution should still stay as DB centric as possible and (a few) times when you just have to do something completely different.

But, as you said at the top, chances are pretty good that's not you. Furthermore, solid state storage (both flash and DRAM based) will be pushing the needle back the other way in terms of performance and will continue to for the foreseeable future. Not saying relational database performance will be better than these nosql products (and the db vendors have some serious catching up to do to use this storage most effectively), but relational dbs will be good enough for an increasing fraction of current use cases.

My predication is that this will cause the nosql movement to quiet down in the next couple years, until there's another significant (faster than storage speed gains) upward spike in the demands we place on systems at which point relational databases will be passe again. Whatever. There's always a race on, but I see no sign that it's time for me to switch horses.

Re: My Thoughts on NoSQL

silverbax — Wed, 19 May 2010 15:44:43 -0000

I agree that 99.9% of all sites are fine with SQL. I 'm also not too impressed with the idea of moving all the data manipulation into the business layer from an application scalability perspective. I guess the question for the NoSQL movement is really, 'what problem are you trying to address'. I am interested in different storage structures, but everything I've tested and read about seems to introduce more divergence and less scalability. The speed of Cassandra is suspect, because well written memory-only SQL manips blow it away. If I'd seen someone at an enterprise company using something like that, I'd think they did not know how to write good DB code and rolled their own hack.

Re: My Thoughts on NoSQL

erikgroenendijk — Mon, 22 Mar 2010 17:23:33 -0000

illuminate's Correlation Database Management System (CDBMS) is a non-relational database engine, driven by the idea that there must be a better way to do ad hoc analysis than trying to make evolved versions of RDBMS/SQL systems faster, better, cheaper by using data warehouse appliances or in-memory, OLAP cubes and column-oriented database structures.

The core of illuminate's correlation database is its unique value-based storage (VBS) structure. Unlike record- or column-oriented data storage structures, in the VBS model, each record is deconstructed during loading to create a unique value, which is stored only once. The extremely compact and fast database automatically creates its own data-generated schema on the fly while loading, which indexes every correlation and provides context to the data values for analysis and reporting. As a result of the VBS approach, a correlation database is optimally structured for exceptional ad hoc query performance.

To build a correlation enterprise data warehouse or data mart, the data itself automatically creates a data-generated schema based on correlations in the data source during the loading process. It's genuine automated database design and best practices. At the same time it's building the schema, every single data value is indexed and stored only once. Need to add new data? Go ahead. The data takes care of itself—no re-architecting, no reloading, no recalculating. Nothing like what relational and column-oriented databases or data warehouse appliances require.

Find out more about the revolutionary illluminate Correlation Database at: www.datainnovationsgroup.com

Re: My Thoughts on NoSQL

Alaric Snell-Pym — Wed, 20 Jan 2010 12:02:48 -0000

It's not so much about being non-relational, though. Relational is a philosophy, rather than something you can definitely point to.

Any relational database that has primary keys can be thought of as a key-value store, and vice versa, for a start.

My company (http://www.geniedb.com/) has made a key-value store with secondary indices. We've then wrapped it up in a MySQL storage engine plugin; our system has no schema, so you can put whatever field=value pairs you want in a record (of any types), but when you create a facade table in MySQL you specify a schema - so when MySQL reads a record from our system it sees NULLs for fields that aren't there, coerces types, etc. And when we update a record, we leave any unknown fields alone. And so on.

If there's a primary key, we use that as the key for the record in our system. If there isn't, we generate our own for each record, and don't expose the key to MySQL as a column; we then use secondary indices and full-table scans to access the table (just as any other storage engine does).

It's not rocket science - but it means that we have our snazzy schemaless replicated distributed database, and people can access it through an API to use all its features; or they can access the same tables through MySQL, trading off performance (MySQL adds a fair amount of overhead) and flexibilty of schema for the ability to use a familiar interface and to use SELECT (with all its lovely joins, GROUP BYs, SORTs, and so on)

Re: My Thoughts on NoSQL

julesmd — Sun, 13 Dec 2009 23:20:58 -0000

This is a great post. i have been looking to get into Django and am strguggling with some of the conepts and methods you are describing here. Are there any other programming tools or tricks that are useful?

Re: My Thoughts on NoSQL

Nati Shalom — Sat, 12 Dec 2009 18:32:33 -0000

Eric
Great and very useful summary.

There are various reports that was gathered from data center statistics that analyze disk failure statistics:
* Actual disk failure/year is 3% (vs. estimates of 0.5 - 0.9%) – this is a 600% difference on reported vs. actual disk failure.
* There is NO correlation between failure rate and disk type – whether it is SCSI, SATA, or fiber channel. * * There is NO correlation between high disk temperature and failure rates

Those analysis shows that the approach of relying on a shared storage for reliability as with most RAC clusters is broken. Instead NOSSQL approach assumes that failure are inevitable and where designed to deal with those failure even under extreme scenarios.

I summarized that topic on one of my recent post: Why Existing Databases (RAC) are So Breakable!: http://natishalom.typepad.c...

You may also want to consider another category of In-Memory-Data-Grid (http://en.wikipedia.org/wik... )

See more details on that regard on Todd Hoff (highscalability.com) write-up: Are Cloud Based Memory Architectures the Next Big Thing? http://highscalability.com/...

Re: My Thoughts on NoSQL

itjob123 — Wed, 02 Dec 2009 13:25:28 -0000

Now I wonder if non-relational databases really offer performance advantages on read/write and write mostly workloads? Or will we end up implementing ad-hoc relational abstraction layers on top of them?

Re: My Thoughts on NoSQL

pppoll37 — Mon, 19 Oct 2009 02:09:28 -0000

mind that a lot of people are in. The sense of wanting to help, but not knowing how or where, is something a lot of us are going through. Please come visit my site San Diego City Business Listings when you got time.

Re: My Thoughts on NoSQL

pppoll37 — Mon, 19 Oct 2009 02:08:03 -0000

mind that a lot of people are in. The sense of wanting to help, but not knowing how or where, is something a lot of us are going through. Please come visit my site San Diego Web Link when you got time.

Re: My Thoughts on NoSQL

pppoll37 — Mon, 19 Oct 2009 02:06:21 -0000

mind that a lot of people are in. The sense of wanting to help, but not knowing how or where, is something a lot of us are going through. Please come visit my site Directory San Diego City when you got time.

Re: My Thoughts on NoSQL

pppoll37 — Mon, 19 Oct 2009 02:05:42 -0000

mind that a lot of people are in. The sense of wanting to help, but not knowing how or where, is something a lot of us are going through. Please come visit my site San Diego Business Directory Forum Blog Classifieds when you got time.

Re: My Thoughts on NoSQL

pppoll37 — Mon, 19 Oct 2009 02:03:53 -0000

Re: My Thoughts on NoSQL

pppoll37 — Mon, 19 Oct 2009 02:01:50 -0000

Re: My Thoughts on NoSQL

Eric Lubow — Fri, 09 Oct 2009 15:55:03 -0000

After spending a bit of time with Tokyo Cabinet both in the hash database and the table database, I wouldn't say that TC table database sucks. But it is substantially slower than the hash database (which is to be expected). In order to compensate for that, I modified the way in which the data was stored (basically serializing the data and dumping it into the value column). The obvious decision here is whether or not you want to add that extra requirement of serialization to your application. I found that it really wasn't much of a performance impact overall to add the serialization step, but my application doesn't require real time process or anything close to it.

My next step is to try a similar thing in Cassandra (without data serialization) and see how it fares. The issue that I think you should have mentioned is that although Cassandra uses the Perl Thrift API, the APIs in general into Cassandra are weak. The Ruby API (through the cassandra gem) is very straight forward though. TC has fairly well documented and straightforward Perl and Ruby APIs. -- http://eric.lubow.org/2009/...

Re: My Thoughts on NoSQL

rr8004 — Mon, 05 Oct 2009 03:19:20 -0000

I enjoyed reading your work! GREAT post! I looked around for this… but I found you! Anyway, would you mind if I threw up a backlink from my site?
Please come visit my site Local Business Directory Of Cincinnati U.S.A. when you got time. Thanks.

Re: My Thoughts on NoSQL

hotlena — Sat, 03 Oct 2009 05:37:50 -0000

Nice posting! I agree with you!

Greetings from Germany

HotLena