Draw a Real Picture

In a previous article I mentioned five tips for Domain Modeling. In this post I'm going to drill down further on one of those:

1. Draw a "Real Picture".

I've used Real Pictures as a technique for a number of years; not only in Domain Modeling, but in Architecture, Design, Development and Planning. It's a pretty simple exercise, but yields a lot of interesting information. In this description I talk in the context of running a project, because that's the most common case -- but that is not universal.

A Real Picture is foremost a conversation piece. It lets the stakeholders involved discuss and explore the domain and it's context. Besides just capturing raw information, a Real Picture also serves as a gentle introduction for participants that might not be familiar with formal diagramming approaches. Often a good first step.

To start, get the participants together. You don't need to mass-invite everyone, but make sure all the key representatives and stakeholders are covered. So if it's a software project, you should have sponsors, end-users, designers, developers, testers and so forth. If you need to economise on people, focus on those that have most affected or impacted by the end result of the exercise, rather than on pure expertise.

There isn't much to developing the picture itself -- Simply draw the major concepts and how they inter-relate. In my experience, this is enough to start the conversation flowing; just draw and re-draw the diagram as the "oh, and then there is" comments evolve.

As a rule of thumb, I keep Real Pictures to one page - A3 or a White Board usually. If it gets really messy, redraw the picture. If necessary, drop in prompters to keep the dialogue flowing. Pick a concept and drill down - focus on quantities, qualities, time, costs, constraints:

- How many of these items are there? e.g. How many employees are at that location?
- Does it vary over the day, are there seasonal variations? e.g. Do you have a rush at Christmas, End of Financial Year?
- Will this change or move soon? e.g. Are you changing network providers? When did you last open a new outlet?
- What would invalidate this element or relationship? e.g. What security issues would make a location unviable?
- What do competitors or other equivalents do for this function or outcome? e.g. Does your competitor do this differently? better?
- Are there constraints on the way things should be? e.g. Does the government enforce strict reporting on certain items?

It's important that you don't constrain the actual process and format too much. The objective of this exercise is to get all the stakeholders together and to articulate a landscape that everyone understands. The end result should also be something that all stakeholders reasonably grasps and agree to.

At this stage, try not to abstract too much - early abstraction can lead to some necessary detail or important corner-case being lost. If a specific location, or server, or piece of software is mentioned, it may have special significance.

In one particular instance I found a number of stakeholders would refer repeatedly to a specific printer - i.e. the actual device itself and where it was located. Most of the project team didn't see any significance of this, myself included. The model and design assumed it was a standard office printer, the same any printer we used in our day to day. When we eventually dug deeper, it turns out this specific instance was special - it enabled secure printing. The way the device functions is such that a human operator never gets to see the contents. If you've ever received a bank PIN in an special envelope, then it's probably from a pretty similar device.

Not only was the fact that this device was special important - but also many of the project participants hasn't realised the significance of the information being send by this device. The communications that ended up at this printer required an extra level of security; it has a very special significance. Up until that point most of the team hasn't considered that special, but clearly it was. Similarly, the end-users thought that information was obvious. Suffice to say, staying concrete at this stage usually is the best default - Abstractions will flow later.

A Real Picture sounds trivial, in fact you might be over-the-top to call it a technique. Either way, it's a low impact exercise with a lot of benefits. At the end of the exercise you have:

• A starter for the vocabulary that is in use. Perhaps more importantly, you've got a vocabulary that a variety participants are at least familiar with.


• An outline of scope. With the Real Picture you should be able to draw a line around what's "in" and what's "out". If something's ambiguous, explore that some more. If scope has been defined as part of a project, this can be a useful validation point.


• A fallback. If you hit a road-block or misunderstanding later, you might be down in some detail and a representation that some might not understand. If you have this, you can ask "but didn't we say these relate in that other diagram?". It's not the absolute source-of-truth, but it's a useful pivot.


• An understanding of the various perspectives of the stakeholders. Usually the Real Picture will help illustrate the natural focus and bias of the stakeholders.


• If the participants can't understand or arrive at a picture, then the composition or overall scope needs addressing. Every participant in the project or domain should at least be able to understand the A3 view of what you're trying to realise.

Jon

Customizing large industry standards

THE PROBLEM

Throughout our consulting engagements with corporates attempting to implement industry standards, two predominant issues keep recurring:
1. Customization of the standard means forking from the standard. Taking updates from this point is at best extremely labor intensive, and at worst impossible.
2. Very few organizations want to implement an entire standard from day one. A more typical usage is a phased approach with core services incrementally released over several years. Absorbing the entire standard up front results in service consumers being expected to understand and work with verbose and confusing generated artefacts (such as XSDs) filled with far more data fields than have actually been implemented. In fact I'd hazard a guess that there are very few cases where more than half of those data fields will ever be implemented.


WHY?

The integration of data and its structure is a major problem for almost all organizations - the bigger the organization the bigger the problem. We have seen many organizations look towards domain industry standards to provide a more structured blueprint for internal and external integration.

Business-to-business standards tend to be fairly effective because each organization involved has a strong financial incentive to adhere to the standard. An example is the AS2805 standard for Australian EFTPOS transactions, which itself was based on the ISO standard 8583.

Internal integration is, however, another story. If you've ever worked in the IT arm of a large organization you'll know what I'm talking about. Almost all software development is project based, and it's an all too common story to hear that the 'refactoring' was descoped to 'phase 2' which - surprise, surprise - never actually ends up being funded.

However, for those lucky enough (or perhaps unlucky enough) to participate in the rare sort of project that does attempt to implement the 'phase 2', you've probably also discovered that adhering to an industry standard is anything but easy. For starters there are few tools available for dealing with standards. Most open standards are available in XSD format accompanied by MicrosoftWord, and MicrosoftExcel documentation, and most proprietary standards are accompanied by equally proprietary vendor lock-in product sales. Beyond this, no standard is ever quite what you need, either missing large areas important to your business, or overly bloated due to a missing-the-point attempt to please all consumers all at once, and more commonly a little of both. Almost all approaches require the consumer to start with the entire standard and then customize for their needs.


SO WHAT?

As practitioners in this space, and still recoiling from the war wounds, these two problems were front of mind in the inception of http://graft.jodoro.com.

Customization

Every model in Graft is internally stored as tiny deltas. When you extend an existing model in Graft you simply add further deltas to the same base structures. Version control and management is baked into the framework of the application. As such, even if you completely rename an existing class, you can easily adopt any future associations to the base class within your extended model. When and what you adopt is controlled automatically via "releases" configured in the 'specify' application.

Templating

In Graft we provide users two distinct ways to extend a model:

1. The most commonly understood approach we call 'active' extension. This quite literally means you start with the entire base model in your scope. Graft then allows users to explicitly descope parts of the model they don't wish to use. Such descoped classes will remain in the default view, but grayed out. This allows users to see what they have removed, and provides the ability to reverse the removal at a later point. Although users can still see grayed out descoped items, such items will not be included in any exports. An example of an actively extended model can be found here: http://graft.jodoro.com/models/31881

2. We are particularly excited to introduce a new approach to extending that we call 'passive' extension. This allows users to begin with everything descoped, and to draw forward only what is of utility right now. This is the approach we strongly recommend be used when working with large industry standards. An example of a passively extended model can be found here: http://graft.jodoro.com/models/31893

Often organizations wish to borrow from more than one standard. Perhaps, for example, the street address structure is particularly weak in their chosen strategic industry standard. Graft allows any number and combination of active and passive extensions within the one custom model. This is again configured in the 'specify' application.

Feel free to extend these and any other public models within Graft. By default extensions of any models will remain private. You can however choose to make your extended models public too by visiting the admin tab.


Doug

5 Domain Modeling Tips

It's been a while since we've updated the blog. In the meantime, we've been working hard -- both consulting in the domain modeling space, as well as preparing the latest release of Graft. Given this, we're going to blog less about the technical implementation - and more about using Graft and the process of Domain Modeling.

To start we thought we'd discuss five key tips for building a domain model. We find these are generally applicable, but how and when you implement them will depend a lot on context. Key factors include the time at hand, pre-existing collateral, requirement maturity and the availability of Subject Matter Experts.

Here is a quick run down for now. We'll flesh these out over the coming weeks:

1. Draw a Real Picture
This has a lot of names - In this case it is just an informal sketch that every stakeholder will understand. Typically this is done on a white-board, but paper or online work just as well. Emphasis here isn't on building a refined model, it's about getting the landscape view, establishing an initial vocabulary and providing a basis of scope.

This picture doesn't have any formal semantics, it functions more as a conversation piece to illustrate and start grappling with the domain. Build this using whatever works for the participants. That said, if done right it can be an opportunity to gently start introducing some key modeling concepts and approaches.

2. Goal-Based Workshops
Many modeling exercises risk becoming esoteric. We encourage goal-based modeling that drives towards outcomes. Highlight the gaps, inconsistencies and pitfalls. Uncover the gnarly pieces early. You might not necessarily solve these immediately, but surfacing them is a key outcome.

Often the Goals themselves are simply driven out through workshops and consultation. In the ideal case you can derive Goals from sources such as strategic initiatives, requirements definitions, existing systems and business processes. If these aren't available or mature, focus on how the outcomes will be validated.

3. Deep-Dives, with a focus on Cohesion and Coupling
This is really where the heavy-lifting of modeling comes in. A deep-dive is a concentrated effort on an important aspect of the domain. Often the Goal-Based Workshops will help to drive out the topics.

The frequency and function of these will need to vary based upon the topic involved. Our ideal is a set of tight iterations on each topic. However, a common compromise is to run a deep-dive session, have the modeler refine the results offline and then a hold final session to reconfirm the outputs.

4. Revisit the Landscape
Deep-Dives generally uncover a lot of detail, however, before going to far down a rabbit-hole it's good to pause and check "is it really relevant?". The key question here is always "what's the context?".

Ideally you will have introduced Business Processes as a requirements driver earlier. However, if this hasn't occurred, this is a key opportunity to introduce them. The Business Process will let you ask a lot of key questions around model attributes and relationships - Are they used? When and what for? This can lead to surprising results, often for aspects the participants take for granted.

5. Iterate, Evolve & Refine
No model is ever "ultimately complete"; in fact it can be damaging to have this as a goal. It's important to plan for how a domain model can and should change over time. This can include small things - from notes and documentation on why decisions have been made and considerations for the future - through to a full roadmap on how the model needs to progress from here.

That's a very quick wrap for now. If you have any other tips, let us know and we'll see about exploring them over the coming weeks.

Jon

Twitter Updates

We'll be publishing more of our release notes through the @jodoro Twitter feed. Particularly now that we'll have more regular releases as we run the Graft Alpha.

Follow Jodoro on Twitter here.

As always, feedback is welcome!

Graft application Alpha release

Jodoro has today released an Alpha version of Graft (http://www.jodoro.com/graft), an online collaborative modelling tool. Graft is publicly available and free to use for those happy to accept that their models will be open and accessible to all other users. Jodoro encourages users to share, collaborate and leverage their models. Future releases of the Graft application will see the introduction of several export formats including xml schemas, and coding stubs for various languages, and a paid offering for users who wish to keep their models and model extensions private. Please feel free to provide feedback, suggestions and thoughts to feedback@jodoro.com.

Thanks, Doug and Jon

Jodoro in the Bay

Doug and I are over visiting San Francisco and the Bay Area for a month or so. We've sublet a place in Inner Sunset, which we'll use as a base to explore San Francisco and the Bay.

Already been a very fruitful trip. In fact, it's initially overwhelming how much there is going on. If anybody has any tips on our trip, feel free to drop me a line.

Looping in Distributed Programming - Redux - Breaks

In Doug's previous article on distributed loops he wrote about three types types of loop - Independent Loops, Accumulative Loops and Dependent Loops. His article goes deeper, but the basic determining factor is the dependency between each loop iteration. This factor is important to us as it directly translates into the level of parallelism we can exploit, and by implication the level of distribution.

As we've worked outwards we've also also started to find other important categorizations - generally under the (broad) banner of an Accumulative Loop. Our main new category centers on what would be called break in C-based languages - or it's often maligned cousins, continue and goto [1, 2].

Break introduces a control dependency between each iteration. In order to know if iteration [n+1] should execute, we first need to know if iteration [n] has decided to break the loop.

This is a bit of a pain, especially in cases where the loop is otherwise independent. The case we're regularly hitting is a simple search where you're only interested a single result.

For example, you might have a function that allows route-planning - maybe planning a series of flights, or bus trips. Rather than calculate some kind of optimum, in the first case you just want to throw together a quick example that the user then tweaks and edits. This might be useful to check if the route is possible at all, before diving into optimization. In this case you're only interested in the first match (if any).

In these cases I can theoretically evaluate a match against each individual entity in parallel. This issue is that after getting my first match, any subsequent iterations are superfluous.

So what are the options. I can:

1. Ignore the redundancy and iterate through the whole set - in parallel and distributed.


2. Build in a control dependency and break the loop when I find a match. This means each iteration must be sequential.


3. Allow a hybrid - Iterate in batches and allow breaks at the batch level.

Clearly the tradeoff here is the amount of redundant processing against the increase in parallelism. If you assume a simple, evenly distributed search space, on average option #1 will do twice the amount of work of option #2. If you greatly increase the size of the search space, this could end up being a lot of redundant work. However, as the search space increases the available parallelism increases also. Naturally it depends, but arguably you get a better "return" on the parallelism side as this has a number of other benefits - e.g. scale, response time and so forth. If you had infinite resources you would certainly adopt approach #1 and work as parallel as possible.

For option #3 you can make the best of the tradeoff if your batches line up well with your distribution model - for example, if you're CPU bound this might be that the batch size lines up with the number of cores available. You're able to exploit the highest level of parallelism that resources allow, whilst considerably reducing the amount of redundant processing.

It's semantically tricky to come up with a construct that makes this construct explicit in our new distribution language. Calling it "break" is the worst as it's overloading an existing term with different, but subtly similar connotations. What we're really after is a succinct way of introducing a "break-loop-at-your-next-convenience" statement. I've settled on "retire" for the moment, but it still doesn't seem to hit the nail on the head (suggestions welcome).

In terms of the technical implementation - Right now we're semantically catering for #3 by introducing control breaks, but under the hood we're implementing #1 by completely ignoring them. #1 is effectively the degenerate case of #3. Once our distribution technology improves we'll be able to exploit these cues better.

jon@jodoro.com

[1] Continue is in the same class generally, but doesn't apply in the specific case being discussed in this article - if you assume an otherwise independent loop, continue can be constructed using if/else semantics.
[2] e.g. http://www.google.com.au/search?q=break+continue+considered+harmful. Some developers put break in the same camp as continue and goto.