ArmRest and the tale of the No Schema Scheme

Posted about 1 year ago by Philip Ingram

I spent three weeks in March of 2010 trying to make CouchDB and Ruby do what i wanted. The idea was simple, i thought:

Have a truly schema-less app.

What I mean by the statement above, is that I wanted to be able, at any time, to add database columns. We have all been told that nosql’s magic allows for this to happen easily, as a document store doesn’t give a fuck what it holds.

Unlike, SQL which needs to be defined and in rail’s case, you need to define the schema in migrations.

Why are Migrations not a good fit?

In this app i was creating, I wanted third parties to define extra “fields” based on industry specific data.

E.g., If you were a restaurant, you’ll have different Inventory categories, then if you are, say, an Automobile Manufacturer.

Making my app as generic as i want, won’t make up for the usefulness of industry targeted input data, that then needs to be calculated later on by the app.

So I thought, let’s go out and see how I can make this happen.

Well, Use NoSQL then…

So reading up on NoSQL sounded really cool. A database that you can change on a moment’s notice and the thing won’t cry! Awesome.

It is true, document databases don’t give a rat’s ass what you put in it. The ORM you choose to use does.

Who Invited the ORM to the party?

Yes, it seems the weakest link is the ORM in my pipe dream of total schmea-less design in my app.

Object-relational mapping (ORM), by it’s very name, suggests schema less design won’t work. You need to know something about the object, of some sort, to relation and map. Since most of the fun things that make Active Record and datamapper cool, all hinge on it’s ability to know “what fields it should expect”. I’m thinking of validations, callbacks, etc.

Your Options, Sire?

Good question Jimmy. In ruby, I found a couple of things that may work:

  • Proxy Objects – This technique is taken from Java, and maybe other hardcore languages like C or PASCAL even (who knows). Your Model ends up with a Class that can handle methods you create for it. Binary Logic has a whole blog post on it. It’s pretty cool stuff, but I think i’d still have a problem using it.
  • Method Missing – The infamous Ruby Kernel method called method_missing allows you to do something to methods in which you didn’t define.
    In the example it used converting numbers to Roman numerals, and rather then dictate what each value should be, it uses method missing to get all meta on it’s arse.
    Now it’s cool and all, but it’s too free for what i wanted, cause i didn’t want EVERYTHING passed in to be “kosher” to my database, only what I said i wanted, shall pass… (Think Gandolf) .

Get to the Solution

There isn’t one in a typical MVC framework. For a solution, you have to hack and slash it to fit. You can try the options above, or here is what i came up with.

Create a new style of MVC framework called TMVC or mMVC. (Template or MiniModel, respectively).

In this style of framework, before your model is called into action, it references a Template, or a Mini Model file, to get all of the variables that it will work with. Storing these variables in YAML or something, makes it easier for non-programmers to contribute, especially if you are dealing with an industry you aren’t familiar with and need these experts input.

Your backend has to be NoSQL, in theory.

Since you are pre loading a schema of endless possibilities, you need a backend that can handle such a thing. I found CouchDB with it’s ACID properties to be a good fit for this. Speed isn’t really a huge need for my app, but saving data reliably is.

Validations happen on Data and field names

To ensure Johnny Crackpot doesn’t XSS you or some shit, you’d have to check that the data is valid. I guess you can have mandatory fields too in the miniModel loading as well. Hell, why not.

Views should be treated like Translations in Rails

In this new world order, systems for loading dynamically generated views need to evolve. Projects like Effigy can be used to load dynamic content based on changing existing field names. I think this only works for css fields or something, but again, all this needs to be adapted anyways, so you can somehow map model fields to view fields similarly however.

Controller methods becomes less used, maybe?

The odd man out is the controller, which really shifts the logic of getting ‘shit’ done to the view. However, maybe this could change with more time and thought.

Conclusion – Introducing ArmRest

While this whole thing is overwhelmingly large, I do feel it has the makings of something worthwhile.

My First crack at starting this is called ArmRest

I started creating a barebones Couch Adapter that relied upon no logic except whatever the CouchDB API required.

What can I do to help?

If you think this project is something you’d like to contribute to, I’m all for it. A lot of what I propose is above my pay grade, as i Know what i know well, but others know more about creating frameworks and the like. Give me a shout and let’s make this thing work.

Feel free to leave critiques, comments, helpful solutions, whatever, in the comments below. Thanks.

Comments

6268be25549e053ddff6f4f2c5315d5f Omar Qureshi
I have a fundamental conflict with having a schema-less schema.

There’s something about the lack of control that makes DocDBs a little scary with regards to not knowing EXACTLY what attributes a record has.

That combined with an overall poor use case for using a DocDB in a majority of cases, if you really go back to the root cause of all these problems it all stems from previously having a MySQL backend.

A lack of understanding of the slightly more advanced features of proper SQL databases.

View (and then later matview) records allow you to have a scalable alternative to a normal database record which seems to have been overlooked by almost everyone mainly because of the guise of “Database Agnosticism” and it gives you a read-only presentation object.

Writing database specific SQL is not a sin. I mean, isn’t that exactly what all these new DocDB adaptors recommend you do?

You could even go a step further by adding constraints to the database to ensure that even if the Ruby based validations don’t catch everything, you don’t lose referential integrity – which is still important to ensure that you don’t need to keep doing horrible checks in your code (e.g. obj.assoc.attrib if obj.assoc).

8f5dac3a09b4b8a3d049be5ad0ba11c7 Philip Ingram
Having been trolling many an IRC chat room with NOSQL folks, this approach really does make you think differently. You almost have to check your SQL knowledge at the door and then it’s liberating.

Now my solutions offered, at least, a different way of using NOSQL with a truly free schema-less approach, by loading a schema defined by another qualified user, early on in the mMVC stack.

Theoretically, it’s not schema less at all, as it gives the flexibility to load the ‘schema’ that your end user would use. Almost the ultimate in UX, i’d say.

No matter how crappy your SQL skills are, or how good, any sql is going to make noise if a field isn’t there. And when you don’t know what fields are going to be there, NOSQL does make sense here. At least couchdb does.

Various people in every country take the mortgage loans from various creditors, just because that’s simple.
If you are willing to buy a car, you would have to get the mortgage loans. Moreover, my sister all the time uses a short term loan, which is really fast.
The life time is very unforeseeable thus, humen do live life in a right way and are trying to get an information connecting to this post. So you know that the job for writers service should assist somebody with this.
Sometimes it occurs that you have no free time to write your thesis sample referring to this good topic. Some different people make a problem from that, just because some of them don’t know about custom thesis service, but I could give you advices about the ways to find the perfect dissertation writing service and the way to buy your dissertation very cheap.
I’m sure that some buy thesis services and dissertation writing service will hire you to complete such kind of fantastic doctoral thesis just about this good post.
Don’t search for the very hot facts as this post in newspapers! Buy essays and buy research paper at the essays writing corporations.
My good friend doesn’t want to write academic papers. He will better buy Research paper ideas in the internet. I don’t know if it is a right decision.
Yeah clearly very cooperative for the readers! Most of the posts in the blog trigger with admirable acquaintance… Thanks for the message! None of the writing services could fit such an excuse, all they can do is to make me buy custom essay papers which are not unrivaled. The only thing you’ve got to see trying to make an definite function is that you can buy term papers or buy research paper about everything.But remember, to make audit about this good post is not an clear task. Thanks.
I’m so happy that ordered high voltage cable testing of electrical equipment as this helped me to detect that there were the problems with their installation.
Mounted the bordered “Garden with Flowes” by Vincent van Cogh on the wall in the living room. Itlooksterrific. Thank your workers for that wonderful oil paintings reproduction they create!
Busty London escorts are not for those dudes who have no desire to waste huge costs. Nevertheless, trust me that they are good value for your money as the girls will give you the great satisfaction.
Busty London escorts are perfect GFE. If you want to enjoy your time, then these ladies will be excellent accompany for you.
Have been seeking such good wholesale hcg proposals for a long period of time. Guys, you did a great job. Can’t wait to receive my order! Believe that I will go along with your company in the future.
{I have {run|had|owned|managed| my own business for more than 10 years. Normally I carry on contructs with my accompaniments in city and must prepare transportation for them. In case you are seeking the agency that serves efficiently
Thank you so much for aiding with programming computer internet. I’m assured that without your help I wouldn’t to overcome this troublesome process.
On this website you will find superlative Holiday rentals Costa de la Luz propositions. My parents rented one country house for couple of months during the vocation and were absolutely satisfied with the choice.
Came across this genetic diseases list accidentally. Was actually informative to hear more info concerning the crucial illnesses of the society.
We may tell for real that this website is an excellent location for a nice Shopping. Guys recommend a huge variety of jazzy clothing, belts, goods for home and garden, etc. Furthermore, you manage order everything at affordable prices and be positive that your purchase will be handed over to you promptly.
Got Electrolux hoover from this shop and I’m entirely plesed with its quality. Furthermore, here you can get original tools for the home and fix the ones that were broken as well.
Hi there! I know this is kinda off topic but I was wondering if you knew where I could get a captcha plugin for my comment form? I’m using the same blog platform as yours and I’m having trouble finding one? Thanks a lot!
Hey there are using Wordpress for your blog platform? I’m new to the blog world but I’m trying to get started and set up my own. Do you need any coding expertise to make your own blog? Any help would be greatly appreciated!
x0MytW comment4
Awesome website you have here but I was wondering if you knew of any discussion boards that cover the same topics talked about here? I’d really love to be a part of community where I can get feedback from other knowledgeable people that share the same interest. If you have any suggestions, please let me know. Kudos!
369bda1854e8a5a5bbc59af38db7d65f Holiday villa Costa de la Luz
It is impossible to imagine more amazing spot than Holiday villa Costa de la Luz to take your time off in. The clime is supreme, background is awesome and the charges are cpmfortable.