Idea - Object Bus

I'm going to blog funky ideas that I have as they occur to me, otherwise I forget them. If you like them so much that you want to use them for yourself, go for it, I'd be honored. Just make sure you don't exclude me from using them also through legal mumbo jumbo, because that's not okay, okay?

Okay, the first one I'll blog is my "object bus", for want of a better term.

Motivation:
I'm an application developer by profession, working in the microsoft environment on mostly database apps, mostly using Borland's Delphi.

I'm also very keen on agile processes, and basically building business/enterprise apps quickly, focussing on the business need and not on the plumbing - really, we've been doing this stuff for long enough that the plumbing shouldn't be an issue anymore.

My default architecture these days for distributed apps (not highly distributed, just stuff that wants a few tiers and a few machines) is to make lots of little pieces, each as simple as possible, which communicate via the database. It works, it's not too bad, but it really is using a database for something that it's not really designed for (communication).

There are many options for interprocess communication and data persistence these days, but it's still a heterogeneous mish-mash, so that if you want to write a conceptually simple distibuted, database backed system, you have to reinvent the wheel a lot, and stuff around trying to work out where your data will live, how the bits of your application will talk to each other, yada yada. What a bore!

Anyway, in that light, I want to create some kind of general purpose "thing" which allows systems composed of many small parts to persists information, communicate (in guaranteed ways), and generally "talk", in a homogeneous fashion, really simply. I think it would enable a level of RAD in n-tiered / distributed business systems which would increase their production & reliability by an order of magnitude, just by simplifying development. In my experience, complex systems lead to many more problems than simple ones.

The Idea:
I want pieces of distributed systems to "talk" on a kind of bus, ie: they put info in, and anyone interested can get it out. This gets broken down into volatile communication (I put info in, someone else signs for it and takes it out, it's delivered and thus gone), and persistent communication (I put info in, it goes into a store of some kind, others access it as desired).

I'm thinking the basic units of communication will be objects, probably encoded in XML. So we are sticking objects onto the bus, and pulling objects off the bus. Actually, no code goes with the objects (the receiver must know what to do with the object) so it's technically just structured data, but that's ok. Maybe a future version would include shipping code around, but I'd like to stear clear of that for now.

Some jumbled points about this bus:
- Callers talk to something local to the machine; the distributed nature of the bus is hidden from them.

- The bus is partitioned into "spaces" (ala spaces in Java); you put your objects into a space, access objects in a space, take objects out of a space.

- Security, at least initially, would happen at the level of spaces.

- Inside a space, you have object collections (just a neutral term for it for now). These are like tables in a relational db, with an associated schema, a name, and a collection of objects that fit the schema. When you put an object in a space, you choose the collection for it to go into.

- The whole thing should be easy from the developer's point of view - it should just go. For instance, when putting an object in, there might be a "create" parameter which causes the requested space and collection to be created (based on the object) if it is not already extant. Things should always lean this way - try to do what the developer wants, don't just throw errors, but tempered with still ensuring that a caller who wants the safety of requiring schemas to be present, etc etc, can get that.

- Under the covers, the system is inspired by p2p systems, each machine talking to which ever other ones can be seen. The machines should still be invisible to the application layer. So adding a new machine to the mix should be as simple as installing the software, then pointing the machine at one already in the system and saying "set yourself up like that one".

- There are probably two basic classes of machine in the p2p environment; clients and servers. Clients are transient members, Servers are permanent members. You wouldn't do anything requiring persistence on a client, for instance. Servers might want a bit more hand setup than clients.

- Persistence might be provided under the covers by a relational db or some such. You should be able to have alternate mechanisms for all required bits of the system, and swap them in/out as desired.

- There might be a one machine only version of the system (good starting point) as well as more complex networked machines. The app should not be able to tell the difference!

- For communicating, many protocols should be available, and the two directions between two machines should be able to be assymetrical (and always asynchronous). To the app this is invisible.

- Spaces and object collections in spaces should be mapable by admins in big setups, so that a space named "x" on one box is equivalent to a space named "y" on another. That way, apps can be hooked together like lego without being designed as such.

- There should be a high tolerance for mismatching schemas in object collections, with every effort taken to squeeze a square block into a round hole. It must just work!

Well, there's a start.