iie labs

ArmRest and the tale of the No Schema Scheme

Posted over 8 years ago by Philip Ingram

I spent three weeks in March of 2010 trying to make CouchDB and Ruby do what i wanted. The idea was simple, i thought:

Have a truly schema-less app.

What I mean by the statement above, is that I wanted to be able, at any time, to add database columns. We have all been told that nosql’s magic allows for this to happen easily, as a document store doesn’t give a fuck what it holds.

Unlike, SQL which needs to be defined and in rail’s case, you need to define the schema in migrations.

Why are Migrations not a good fit?

In this app i was creating, I wanted third parties to define extra “fields” based on industry specific data.

E.g., If you were a restaurant, you’ll have different Inventory categories, then if you are, say, an Automobile Manufacturer.

Making my app as generic as i want, won’t make up for the usefulness of industry targeted input data, that then needs to be calculated later on by the app.

So I thought, let’s go out and see how I can make this happen.

Well, Use NoSQL then…

So reading up on NoSQL sounded really cool. A database that you can change on a moment’s notice and the thing won’t cry! Awesome.

It is true, document databases don’t give a rat’s ass what you put in it. The ORM you choose to use does.

Who Invited the ORM to the party?

Yes, it seems the weakest link is the ORM in my pipe dream of total schmea-less design in my app.

Object-relational mapping (ORM), by it’s very name, suggests schema less design won’t work. You need to know something about the object, of some sort, to relation and map. Since most of the fun things that make Active Record and datamapper cool, all hinge on it’s ability to know “what fields it should expect”. I’m thinking of validations, callbacks, etc.

Your Options, Sire?

Good question Jimmy. In ruby, I found a couple of things that may work:

  • Proxy Objects – This technique is taken from Java, and maybe other hardcore languages like C or PASCAL even (who knows). Your Model ends up with a Class that can handle methods you create for it. Binary Logic has a whole blog post on it. It’s pretty cool stuff, but I think i’d still have a problem using it.
  • Method Missing – The infamous Ruby Kernel method called method_missing allows you to do something to methods in which you didn’t define.
    In the example it used converting numbers to Roman numerals, and rather then dictate what each value should be, it uses method missing to get all meta on it’s arse.
    Now it’s cool and all, but it’s too free for what i wanted, cause i didn’t want EVERYTHING passed in to be “kosher” to my database, only what I said i wanted, shall pass… (Think Gandolf) .

Get to the Solution

There isn’t one in a typical MVC framework. For a solution, you have to hack and slash it to fit. You can try the options above, or here is what i came up with.

Create a new style of MVC framework called TMVC or mMVC. (Template or MiniModel, respectively).

In this style of framework, before your model is called into action, it references a Template, or a Mini Model file, to get all of the variables that it will work with. Storing these variables in YAML or something, makes it easier for non-programmers to contribute, especially if you are dealing with an industry you aren’t familiar with and need these experts input.

Your backend has to be NoSQL, in theory.

Since you are pre loading a schema of endless possibilities, you need a backend that can handle such a thing. I found CouchDB with it’s ACID properties to be a good fit for this. Speed isn’t really a huge need for my app, but saving data reliably is.

Validations happen on Data and field names

To ensure Johnny Crackpot doesn’t XSS you or some shit, you’d have to check that the data is valid. I guess you can have mandatory fields too in the miniModel loading as well. Hell, why not.

Views should be treated like Translations in Rails

In this new world order, systems for loading dynamically generated views need to evolve. Projects like Effigy can be used to load dynamic content based on changing existing field names. I think this only works for css fields or something, but again, all this needs to be adapted anyways, so you can somehow map model fields to view fields similarly however.

Controller methods becomes less used, maybe?

The odd man out is the controller, which really shifts the logic of getting ‘shit’ done to the view. However, maybe this could change with more time and thought.

Conclusion – Introducing ArmRest

While this whole thing is overwhelmingly large, I do feel it has the makings of something worthwhile.

My First crack at starting this is called ArmRest

I started creating a barebones Couch Adapter that relied upon no logic except whatever the CouchDB API required.

What can I do to help?

If you think this project is something you’d like to contribute to, I’m all for it. A lot of what I propose is above my pay grade, as i Know what i know well, but others know more about creating frameworks and the like. Give me a shout and let’s make this thing work.

Feel free to leave critiques, comments, helpful solutions, whatever, in the comments below. Thanks.

Comments

blog comments powered by Disqus